Success with GPUImage

[JAndo]

Hello,

I just got my hands on the r200 device. Are any of you using GPUImage with the R200 or R300? If so can you please share your project setup? I am having trouble including librealsense in my swift project.

I have made a modeule.map file and still no luck.

Thanks

[MartyG · Accepted solution]

My research did not find anyone using GPUImage with the RealSense camera, sadly. GPUImage is apparently meant for use with devices using Apple's iOS OS though. Are you intending to use the R200 as an ordinary webcam for developing a GPUIMage application, and not use any of the RealSense features in the camera (depth, infrared, etc)?

In a recent case, a user was unable to get their R200 to provide RGB video to Skype. Research in that case determined that as the R200 is designed as a rear-facing camera (like the camera on the back of a tablet) rather than one that faces towards the user like a usual webcam, it may not be recognized as a usable RGB webcam. In Skype, it could only use the computer's ordinary webcam as the RGB video camera, and recognized the R200 only as a secondary rear-facing camera that could be toggled to provide an alternate camera view. It could not be used on its own as the primary RGB video camera though.

Regarding using Swift with Librealsense ... I could fnd only one other person asking how to use Swift with the RealSense camera, and unfortunately they had not received an answer to their question. I am guessing that given GPUImage's iOS nature, you are using Librealsense on a Mac?

Although Swift is focused on developing applications for Apple devices such as iOS and tvOS, I note that it can also be used with MacOS, and so may be compatible with Librealsense in that respect.

[JAndo]

Marty,

Thank you very much for the quick and thorough response. I am very familiar with GPUImage and that is why it is my graphics library of choice. What is new to me is the Intel camera technology. I was hoping that the camera was sending an RGBA feed to the computer. Since anything a webcam captures is only RGB, it leaves the alpha channel free to send depth information as a greyscale. It sounds like that is not how the camera works and thus might not be the right solution for needs. Is there any way to get the RGB and depth information from the camera in a single image by taking advantage of the alpha channel?

Thanks

[MartyG]

Further research confirmed that the R200 can transmit RGBA. The R200's information page states "The color cam is capable of doing 32 bit RGBA at 1080p @60FPS using fixed focus and a 16:3 aspect ratio. The RGB cam has a slightly larger FOV than the dual cams but is not meant to be used as a standalone camera."

https://software.intel.com/en-us/articles/realsense-r200-camera Introducing the Intel® RealSense™ R200 Camera (world facing) | Intel® Software

Whilst this knowledge is useful, it does not directly help you to generate an RGBA channel on a Mac, since the RealSense SDK for Windows does not have a Mac counterpart.

One approach to RGBA conversion that would work with Librealsense may be to convert the RealSense image into a .mat format image via matlab. Example:

https://stackoverflow.com/questions/30855352/rgba-image-from-intel-realsense-to-matlab RGBa image from intel realsense to matlab - Stack Overflow

I also came across an application for RealSense called RealSense TOP thay can generate RGBA images.

https://www.derivative.ca/wiki088/index.php?title=RealSense_TOP RealSense TOP - TouchDesigner 088 Wiki

TouchDesigner has a MacOS version.