OpenCV and D435

[PG8]

Hi everyone,

Got a D435 - awesome product!

Only problem is I've been having trouble using it beyond the RealSense Viewer.

I have an OpenCV C++ project in eclipse on Ubuntu 16.04. When I plug in my camera (to a USB 3.0 port) and run my project, OpenCV throws a "Pixel format of incoming image is unsupported by OpenCV". When I add the rs2.hpp header to start learning how to convert RealSense data to OpenCV, eclipse throws me 346 errors about unrecognized symbols and others in various RealSense library files.

How do I convert D435 output to OpenCV?

[MartyG · Accepted solution]

Have you read the Getting Started guide for RealSense and OpenCV?

https://github.com/IntelRealSense/librealsense/tree/master/wrappers/opencv librealsense/wrappers/opencv at master · IntelRealSense/librealsense · GitHub

[PG8]

HI MartyG

Thanks for the quick reply. I just took another look at it. I have taken all the steps previously.

I did some research and it appears that the RealSense uses the Z16 pixel format, which is for depth. OpenCV doesn't appear to support Z16 format, but supports Y8. Do you have any suggestions on how to convert this?

I'll try reinstalling RealSense, the library is still throwing errors.

[MartyG]

The best approach to start with is likely to be to get OpenCV and RealSense SDK 2.0 set up first and then test it with the ''Imshow' sample, which is a very basic program to test that OpenCV is working with the camera.

https://github.com/IntelRealSense/librealsense/tree/master/wrappers/opencv/imshow librealsense/wrappers/opencv/imshow at master · IntelRealSense/librealsense · GitHub

[PG8]

Thanks. Tried reinstalling and same issue. A bunch of undefined references in the LibRealSense files (e.g. rs_frame.hpp & more). This is with Imshow

[MartyG]

Reading your case again from the beginning, I'm wondering now whether it is the use of Eclipse that is causing the undefined reference problems. I believe you may need to configure Eclipse to work with CMake.

http://johnnado.com/use-cmake-with-eclipse/ Build with CMake in Eclipse - John McGehee

[PG8]

I figured out the problem, it's that the RealSense files (like rs_context.h) only have the header files and no .c files for definition. When I try to call something like rs_create_context, it can't find a method for it and hence a definition. Is LibRealSense supposed to have .c/cpp files for these files?

[MartyG]

Unfortunately I have no reference for setting up the SDK using Eclipse. The standard method to set it up is to use CMake. I can only recommend waiting for a member of the Intel support team (of which I am not a member) to offer advice. I apologize for the inconvenience.

[PG8]

Hey, no worries, I appreciate the help regardless.

If it helps, it's not an eclipse issue as I'm having the same issue running g++ in terminal on Ubuntu 16.04 LTS (same machine, eclipse is just the IDE).

[PG8]

Got it working, managed to run it and everything, all I had to do was change the command in terminal. But now I'm getting an error for Imshow "frame didn't arrive with 5000". Is this a RealSense error or OpenCV error?

[PG8]

Nevermind, bad plug. Issue is resolved (for now). Thanks for the help!

[MartyG]

Awesome news, thanks for letting us know. Good luck with your work!

[CCarl8]

Hi. I am having a similar issue now, with OpenCvForUnity. Reading the depth image byte[] array using mat.put(). CvType.CV_16UC1 does not seem to be supported. I get the error "Mat data type is not compatible". Is there a way to convert the bytes before reading?

[MartyG]

It may be best to post your question to the creator of the OpenCV For Unity package. You can contact them here:

https://enoxsoftware.com/opencvforunity/contact/technical-inquiry/ https://enoxsoftware.com/opencvforunity/contact/technical-inquiry/

[CCarl8]

Thanks, I just wrote them. Let's see what they say...