Undistorting 415 colour images

[JStev1]

Hi,

Although 415 image distortion coefficients do not seem to be calculated (seem to be all 0), I have been trying to calculate these myself. If these are known, is there a way to get an undistorted RGB image through librealsense? To explain my use case: I created a mesh from all my depth images. Now I want to texture it with the colour images. But I am getting a shift on my mesh. I presume this may be due to the RGB image being projected is not undistorted. I have tried to undistort the images using the left/right distortion coefficients in OpenCV (since these may be quite similar) but my code is just crashing on using those values. I remember reading somewhere that the coefficients were a 'modified' brown conrady model. Is this a model which is used just by RealSense? Or is it a common model.

[MartyG]

Yes, D415 uses Modified Brown-Conrady.

Others have modified (or 'extended') the Brown-Conrady model, suggesting that whilst the Modified Brown Conrady model may be unique to RealSense, modified models in themselves are not unusual.

https://www.researchgate.net/publication/43067483_Modeling_of_radial_asymmetry_in_lens_distortion_facilitated_by_modern_optimization_techniques Modeling of radial asymmetry in lens... (PDF Download Available)

[JStev1]

Is there any info on what modifications are made for RealSense? So that I can translate this model into a more conventional one that I can undistort with OpenCV.

[MartyG]

A technical paper that had one of the 400 Series camera's engineers as an author says about the modified model: "The modification comes in the form of computing tangential terms with radially-corrected x, y. The open-source library for accessing the Intel RealSense cameras details the distortion model exactly".

Here is a link to the full paper:

https://arxiv.org/pdf/1705.05548.pdf https://arxiv.org/pdf/1705.05548.pdf

Digging through the Librealsense code for Modified Brown-Conrady formulas, the closest I found to such a thing was the code below that is applied when Modified Brown-Conrady is the distortion mode.

const r2 = x * x + y * y;

const f = 1 + intrinsics.coeffs[0] * r2 + intrinsics.coeffs[1] * r2 * r2 +

intrinsics.coeffs[4] * r2 * r2 * r2;

x *= f;

y *= f;

const dx = x + 2 * intrinsics.coeffs[2] * x * y + intrinsics.coeffs[3] * (r2 + 2 * x * x);

const dy = y + 2 * intrinsics.coeffs[3] * x * y + intrinsics.coeffs[2] * (r2 + 2 * y * y);

x = dx;

y = dy;