Can I use multiple Intel RealSense cameras in a single Unity3d scene?

[MZaid]

I want to be able to use multiple cameras in the same Unity3d scene. While I was going through the SDK I saw the RsDevice.cs is a singleton so I'm not sure how multiple cameras will work in the same scene.

Any help would be appreciated.

[MartyG · Accepted solution]

Using multiple cameras in the same Unity scene certainly is possible, and has been done. A user called FrenchTouchXR worked out a way to get around the singleton problem and record multiple streams by making edits to the Unity wrapper's scripting, and provided a YouTube video demonstration (the audio is in French though).

Edit for recording multiple streams

https://github.com/IntelRealSense/librealsense/issues/2203#issuecomment-412218681

YouTube video of multiple camera recording in Unity

https://www.youtube.com/watch?v=Zm8ZjLsw-vs

The code of the RealSenseConfiguration.cs file that frenchtouchXR's method edits

https://github.com/IntelRealSense/librealsense/blob/7148f9ae1d78b5d44bee4fc578bf0b8fb9a220c5/wrappers/unity/Assets/RealSenseSDK2.0/Scripts/RealSenseConfiguration.cs

[MZaid]

Hey thank for those links. I see he has a solution for recording from multiple devices. He says he fixed live streaming for multiple as well but no code for that anywhere :(

[MartyG]

I had looked for the setup information myself during researching of your question but was unable to find it. I recall during the initial conversations with FrenchTouchXR before he worked out his multicam solution, an old unofficial wrapper with multi-cam streaming was being looked at. A problem with this wrapper was that it became increasingly out of date as the official Unity wrapper's development progressed. I believe it was FrenchTouchXR's hope to take the bits that he needed from the unofficial wrapper and integrate them into the official one to get the best of both worlds.

https://forums.intel.com/s/question/0D50P0000490SEPSA2/unofficial-unity-extension-for-realsense-400-series?language=en_US

[MZaid]

Cool thanks a lot.. I'll have a look at this wrapper.

I was thinking i'll probably just remove the singleton and because I just need the RsStreamTextureRenderer i'll pass it a reference of RsDevice.

Thanks a bunch !

[JMcCl6]

i know this is old but it may be useful to someone.

since I couldn't find a specifically helpful answer anywhere, here's what you do. 

(Worked for me anyway In unity 2019.3.1f1)

1) create one rsdevice and get it working and duplicate it for each sensor you want to add and provide the serial number for each. 

keep the original for now for reference. 

2) create a new mat, shader and processingpipe for each (just duplicate the originals)

3) leave stream index at 0 for all. 

4) swap out the mat and it's shader and processingpipe in each of the duplicate rsdevices. 

5) for each device pay attention to the number and type of profiles. 

  for example, if you don't use IR remove it from the array of profiles for that device in the editor. 

6) disable/remove the original rsdevice you duplicated (or keep it as the first device)

7) you may have to lower the frame rates if you have several devices. 

I didn't modify any code or have to sync the devices and it worked for me. 

good luck and have fun!

[MartyG]

@JMcCl6​  Thanks so much for your great guide! I have reposted it in full on the current RealSense forum for ease of access. :)

https://support.intelrealsense.com/hc/en-us/community/posts/360047049173

[visual]

@JMcCl6  Hi, does this method still work in the latest version of Unity? I'm a bit unsure if I did steps 2 and 4 correctly.