Depth data for D415 in a range between 1m to 5m is very noisy and unreliable

[bazinga]

Hi,

I'd like to use the D415 camera to perform an estimate of human movement in a area of 1m to 5m in front of the camera. However I am really struggling with setting up the right parameters. I have read many of the posts and github issues, but am still quite disorganized regarding what to change / to do.

I have 2 main issues:

(1) The depth data has a tendency to "jump", i.e. there is much flickering noise in the data starting already at a range of maybe 2m.

(2) [more severe]: In some areas it often looks like the surface is pulled into the background along the ray that hit the camera. The further away the area is the more severe it gets. I have tried different settings, but the default one seems to work best for me (and is used during capturing the attached image). High accuracy seems like no option as the fill-rate for objects further away than 2m is decreasing way to much. I have a feeling that the D415 is not that suitable for this application, however I still think that with the right settings the result should be a lot better than what I am getting..

The viewing angle is sort of from the side to show how the depth data is pulled away. In the front is a robot and a char (right). The whole scene itself is actually surrounded on the left and in the back by a banner-like decor thing so there is a flat surface that is not visible at all.

I was hoping to get some ideas and input in what direction I should go with the parameters since I cannot get this in control by myself..

Thanks for any advice and help!

[MartyG]

Could you tell us first whether you have fluorescent lights such as ceiling strip-lights in the location where you are using the camera? These can create disruption in the image because the gas inside the lights flickers at a frequency that it hard to see with the human eye. If you do have fluorescent lights, operating the camera at a frequency closer to the operating frequency of your particular set of lights may help. for some people this is 30 FPS, and for others it is 60 FPS.

If the image that you have provided is a point cloud, stretching of the image when viewing the point cloud from the side is a known phenomenon with the 400 Series cameras. There was a discussion about this in a case last month in April 2019 (linked to below).

https://forums.intel.com/s/question/0D70P000006H82gSAC

[bazinga · Accepted solution]

Hi,

I have tested the camera at different frame rates with no improvement. However while doing so I have noticed that the depth data looks much better on lower resolutions and best at 640x480. The official whitepaper for tuning does not recommend that resolution. Especially with the high resolution preset it is a very big difference where a chair in front of the camera is barely visible at a resolution of 1280x720 but almost perfectly visible at a resolution of 640x480.

But my main problem is still the "stretching". I have read through the thread that you have linked but it did not help me. I have attached some other images of a scene, to maybe show better what results im getting and what is wrong with them.

The first image is captured from rviz where I stream the pointcloud with the official ros realsense node. The second images are the 2d and 3d perspective of the same scene out of the realsense-viewer tool.

The read box in the first images marks actually a flat surface.. as you can see in the background it's by far not where it should be!

In the post you have linked a github issue is mentioned: https://github.com/intel-ros/realsense/issues/507 When I compare depth quality to the one the user in the github issue has on his images I have to say that I am way off. It looks to me like some major setting is broken. At this point the data is almost unusable because a given pixel with depth d can jump much closer to the camera within the next following frame and back again within the next..

The depth data really only has acceptable quality while being very close (maybe around 1m) to the camera so far. Id really like to use it for much greater ranges tho!

[MartyG]

Typically, the accuracy of depth data starts drifting noticeably when the observed object is 3 m or more from the camera, due to a phenomenon called 'RMS Error' where accuracy reduces over distance. Other factors in the environment can be disruptive to image quality though.

Your mention of greatly fluctuating depth reminded me of a case from March 2019 where a user was experiencing similar changes in depth from one frame to the next. Their flat surfaces such as walls also looked like a "storm sea". The user found that turning off auto-exposure and setting exposure manually helped image quality a bit.

https://forums.intel.com/s/question/0D50P00004HnOsySAF/inconsistent-depth-using-d435i