topic Re: Real time coordinates (x,y,z) with opencv and realsense sdk in Items with no label

Real time coordinates (x,y,z) with opencv and realsense sdk

idata — Sat, 29 Sep 2018 04:25:57 GMT

Hi guys,

I'm new to intel realsense and would appreciate anyone pointing me to any relevant information. I'm using a D435 but it hasn't arrived yet so I haven't been able to try any of the realsense sdk examples. I've created a proof of concept program that can detect and track a new object that has entered into my current webcam's view using opencv. I've also extracted the x,y coordinates and am hoping that the the D435 will provide the z coordinates. Is there anyway to implement the realsense sdk with opencv such that it will give me the z depth of the object that i'm currently tracking?

Also, is there anyway to set the maximum distance that I can track an object, eg, if an object is more than 10m away, I use the realsense sdk to prevent any detection of moving object in opencv.

Sorry if this question has been asked before. I appreciate any answers and anyone pointing me to any relevant materials.

Re: Real time coordinates (x,y,z) with opencv and realsense sdk

MartyG — Sat, 29 Sep 2018 06:37:47 GMT

The RealSense SDK 2.0's 'RealSense Viewer' program can load in pre-recorded data files so that you can practice with the software without having to have a camera attached. You can download the sample data from the link below.

https://github.com/IntelRealSense/librealsense/blob/master/doc/sample-data.md librealsense/sample-data.md at master · IntelRealSense/librealsense · GitHub

OpenCV is not required to detect Z-depth with the D435. If you have already written a program using it though then it is understandable that you would wish to continue with that. The developer UnaNancyOwen has created a depth sample that uses OpenCV.

https://github.com/UnaNancyOwen/RealSense2Sample/tree/master/sample/Depth RealSense2Sample/sample/Depth at master · UnaNancyOwen/RealSense2Sample · GitHub

There are also a couple of object detection examples for the RealSense 400 Series cameras that use OpenCV.

https://github.com/IntelRealSense/librealsense/tree/master/wrappers/opencv/dnn librealsense/wrappers/opencv/dnn at master · IntelRealSense/librealsense · GitHub

https://github.com/twMr7/rscvdnn GitHub - twMr7/rscvdnn: Test program for OpenCV DNN object detection with RealSense camera

In regard to a maximum distance, setting up a 'MaxZ' depth clamp may work for you.

https://github.com/IntelRealSense/librealsense/issues/2199 Set depth clamp min/max from C++ and python code · Issue # 2199 · IntelRealSense/librealsense · GitHub

Re: Real time coordinates (x,y,z) with opencv and realsense sdk

idata — Sat, 29 Sep 2018 08:13:54 GMT

Hi thank you so much for all the resources.

I would like to rephrase so my first question is clearer. I'm trying to detect a black pea that enter my camera's view and capture it's real time x,y,z values. I don't need to classify it in any way. So what you're saying is that I can do my program 100% in the realsense sdk?

Re: Real time coordinates (x,y,z) with opencv and realsense sdk

MartyG — Sat, 29 Sep 2018 08:41:44 GMT

You could certainly do the depth sensing part with the RealSense SDK only. Detection and recognition of objects would usually require third-party software such as OpenCV though. Here's an example YouTube video of a face's motion being tracked in OpenCV using a point cloud.

https://www.youtube.com/watch?v=4fJCAQZpvKk Face Point Cloud Tracking using PCL, OpenCV, and Kinect - YouTube

The only way I can think of to detect the mosquito with the RealSense SDK alone would be to program an application to detect a particular color in the IR image that the mosquito always shows up as, and trigger a 'detect' event if that color is recognized in the IR image.

As a mosquito is a living creature with blood under its skin, red may show up best. It would be even easier to detect once the mosquito has drank blood and stored it inside itself.

In the older models of RealSense camera, a technique called Pulse Estimation was demonstrated to estimate heart rate based on the color of blood under the skin on the IR image.

https://software.intel.com/en-us/articles/pulse-detection-with-intel-realsense-technology Archived - Pulse Detection with Intel® RealSense™ Technology | Intel® Software

Re: Real time coordinates (x,y,z) with opencv and realsense sdk

idata — Sat, 29 Sep 2018 09:07:13 GMT

Thats amazing. I can't thank you enough for all the help. I will definitely look through everything in more detail. I'll get started on detecting a small object in opencv first and getting the depth information and work my way up from there. Thank you so much once again

Re: Real time coordinates (x,y,z) with opencv and realsense sdk

MartyG — Sat, 29 Sep 2018 09:11:48 GMT

You're very welcome! Please feel free to come back here to the forum with further questions whenever you need to. Good luck!