Why does the sensor D435i give bad results in 3D recostruction?

[LScip]

I used the D435i sensor to do 3D Reconstruction with ElasticFusion. While with other sensors the results are pretty good, with this sensor there are some problems, especially with flat surfaces. They seem to have "bumps" which are quite visible on the floor and on the desk. Can you explain why?

[Eliza_D_Intel]

Hello LScip, Thank you for your interest in the Intel RealSense D435i camera. Is it possible to calibrate your device? You can find the Calibration Tool here: https://downloadcenter.intel.com/download/28517/Intel-RealSense-D400-Series-Calibration-Tools-and-API?v=t Also, you mentioned that you do not see the same behavior with other sensors. What are the cameras that you have previously tried? Thank you and best regards, Eliza

[LScip]

I have already calibrated the device using that tool. Another sensor that I used is the Primesense.

[Eliza_D_Intel]

Hello LScip, A good resource for resolving this problem can be the Best Known Methods of Tuning RealSense cameras: https://www.intel.com/content/dam/support/us/en/documents/emerging-technologies/intel-realsense-technology/BKMs_Tuning_RealSense_D4xx_Cam.pdf Thank you and best regards, Eliza

[LScip]

I've been playing around with the Depth Quality Tool as explained in the document, but the result doesn't change. The reconstruction is not bad, but the surfaces are still very bumpy.

[MartyG]

The tuning guide says that bumps can be generated by depth noise, and recommends trying the 'Edge-preserving' post-processing filter to smooth them out. Please left-click on the extract below to view it in full size.

[LScip]

Thank you for your answer. Yes, i read this part concerning the spatial filter. But it didn't change much.

[MartyG]

I wonder if the bumpy areas correspond to the areas of the room on the RGB image where there appears to be strong blue illumination, and black empty areas on the floor where those illuminations reflect off it. Reflections can result in a degradation of image quality in those areas, as the reflection reduces the ability of the camera to analyze those spots properly.

[LScip]

I'm quite sure this is not the case. Experiments were conducted during different moments of the day, with different illuminations, but the results are all pretty similar.

[MartyG · Accepted solution]

If you are confident that the illumination is not the source of the problem, another cause could be the 'IR projector' component of the camera. Its function is to project a pattern of semi-random dots onto a scene to add analyzable texture to low-detail surfaces, such as floors and walls (and desks). On laser-based projectors like the one that is built into the camera, an effect called 'laser speckle' can adversely affect depth.

The tuning guide says that you may get around 30% better on the 'RMS error' (depth noise) with the projector turned off. If the location you are scanning in is brightly lit then you may not need the projector enabled anyway. Unlike other depth cameras, the 400 Series cameras perform better in strong light, "bringing out" the texture of objects.

It looks in your images as though you are generating a textured point cloud. Is that correct, please?