Tracking and Pose detection of moving object

[SKuma51]

Hi,

I'm using Intel D415 for tracking and pose detection of a moving object in the vicinity of 5-6m range. Any idea how accurate it could be and how to get the pose of the tracked object in ROS or OpenCV?

[MartyG · Accepted solution]

The D415 has less depth noise (or 'RMS error') in its image over distance than the D435 model does. On the other hand, the D415's 'rolling' type shutter is slower than the D435's 'global' type shutter, so fast moving objects may cause a smear in the object. A way to counter this if it occurs may be to change the exposure value, as shutter speed is also known as 'exposure time'.

The depth noise over distance of the D415 compared to D435 is shown in the chart below. the D415 is represented by the lower green curve.

If you want to track the position of moving objects, I wonder if a SLAM system would be an appropriate solution. I would recommend the OpenCV-based ORB-SLAM2.

https://github.com/raulmur/ORB_SLAM2 GitHub - raulmur/ORB_SLAM2: Real-Time SLAM for Monocular, Stereo and RGB-D Cameras, with Loop Detection and Relocalizati…

[SKuma51]

Thanks for your quick reply. Yeah SLAM would be handy but i was thinking of getting the pose from the image itself like make some markers on the object and try to get the position and orientation through the disparity maps. So, if I´m doing a blob tracking (for example 2 ping pong balls), can we calculate pose of the ping pong ball using point cloud or any other method?

[MartyG]

There is an unofficial extension for RealSense SDK 2.0's Unity wrapper that supports using ArUco markers on objects to get position and pose estimation. The extension is out of date now compared to the official wrapper, so using it means that you don't have access to the newer features in the official wrapper. I know of a case though where someone took out of the unofficial wrapper the part they wanted and integrated it into the official Unity wrapper.

If you do not wish to go the Unity route, there is an ArUco library for ROS.

http://wiki.ros.org/aruco aruco - ROS Wiki

[SKuma51]

Yeah, i would try that as my last resort. Anyways Thanks.

[MartyG]

Another example I can think of is one where OpenCV was used to track a face with a point cloud using OpenCV's Haar Cascade Classifier.

https://www.youtube.com/watch?v=4fJCAQZpvKk Face Point Cloud Tracking using PCL, OpenCV, and Kinect - YouTube

[BBavi]

Another example I can think of is one where OpenCV was used to track a face with a point cloud using OpenCV's Haar Cascade Classifier.

https://www.youtube.com/watch%3Fv%3D4fJCAQZpvKk Face Point Cloud Tracking using PCL, OpenCV, and Kinect - YouTube

[SKuma51]

Yeah but probably was thinking of making markers on object and then try to get the orientation.