D415 depth data of a small ball

[SFisc6]

Hi,

I'm trying to get the depth data of a small ball. The ball should be in the red square (attached image). The ball is a about 2mm in diameter.

But the best depth data i can get is no data.

Any suggestions on how i can get valid data ?

[MartyG]

I can relate to this concept, as I once gave advice on a golf ball project where they wanted to use a handheld device with a camera to scan the distance between the ball and the golf cup. The ball scale is not the same, though the basic principles of distance-reading likely are. Currently, they needed an expensive 17 foot tall tower on the golf green to perform the reading.

A 2 mm size ball will be hard for the camera to see with a depth scan when viewed from a distance. In that golf project case, I suggested capturing an RGB image and finding the coordinates of the ball on the RGB image, converting them from 2D pixel coordinates to 3D world coordinates.

RealSense SDK 2.0 has a couple of sample programs for recognizing an object from an RGB image using DNN (Deep Neural Network) principles, and then getting a depth reading

https://github.com/IntelRealSense/librealsense/tree/master/wrappers/opencv/dnn

https://github.com/twMr7/rscvdnn

[SFisc6]

hm, the recognition isn't the problem i got that covert.

The distance isn't even that great, max 80 cm.

But how do i convert 2D pixels in accurate 3D world coordinates without the depth data ?

[MartyG]

A depth camera already has trouble reading spheres because of the lack of complex surface detail on them. At 2mm, the size of your ball likely does not have enough surface detail for the camera to read from 80 cm away. On the images that you supplied, the ball just merges in with the rest of the ground.

Can the camera detect depth for the ball if you move the camera in closer?

[SFisc6]

No. The best i got until now is a consistant black point (i added light from above) where the ball is located. Maybe i can get better results with a different parameter set. Have you got any suggestions on what parameters i should modify ?

[MartyG]

A black area can be symptomatic of the camera not being able to read detail in that area. This can particularly occur if the object is reflective or if it is a dark grey or black color. For example, if a black cable is depth-scanned then it typically appears on the image as a cable-shaped solid black line. This is because dark objects like these absorb the camera's light.

If your ball is not dark but it is reflective, adding additional light may make the ball part of the image worse. You may be able to dampen the reflection for a better quality scan by spraying a fine powder such as foot powder or baby powder onto the ball. There are also professional 3D scanning aerosol sprays available.

[SFisc6]

Reflection and color shouldn't be a problem. It's a white styrofoam ball

[MartyG · Accepted solution]

The primary problem is likely to be the size of the ball and the distance of the camera from the ball. In this situation there may not be many options for depth-reading the ball successfully.

The next thing I would try is altering the 'depth units' of the camera. The 400 Series cameras are set up for long-range scanning, but this means that at close range the results may not be as good as the short-range SR300 camera from the previous RealSense generation. It is possible to alter the depth units of the 400 Series camera to give a close-range performance more comparable to that of the SR300.

The depth units dictate how far the camera can see, or its 'expressive range'.. The default depth unit on the 400 Series cameras is 1mm, meaning that it can see for a distance of 65 meters. However, this long distance range gives it a disadvantage at close-range scanning. The older SR300 camera model has a default scale of 1/32th of a millimeter, allowing for a maximum depth sensing range of two meters but giving it better depth sensing accuracy at close range.

One way to think of it would be people with short-sighted and long-sighted vision. Short sighted people (represented by the 400 Series camera) can see long distances but not see close-up detail so well. Long sighted people (the SR300) can see close-up details well but cannot see far-away objects - the SR300's maximum depth sensing range is around 2 meters. By changing the depth units of the camera, you can change how short-sighted or long-sighted its view is.

An alternative, simpler way to get better close-range performance is to use a lower resolution.