Realsense Python API for transforming pixel to world coordinates

[YLitv]

Hi,

As part of my research, I am using the D415 Realsense camera to capture depth images of an object randomly placed on a table (I am using only depth images). I used image detection to find the exact pixel in the depth image representing the center of the object, and I would like to know how to transform from that pixel position (row, column) in the depth image to the real Cartesian position (x,y) in the world, where the real depth and z are known (the images are always being taken from a certain known position above the table). I work with Python on Ubuntu 16.04. I looked at https://github.com/IntelRealSense/librealsense/wiki/Projection-in-RealSense-SDK-2.0 Projection in RealSense SDK 2.0 but couldn't find my answer.https://github.com/IntelRealSense/librealsense/wiki/Projection-in-RealSense-SDK-2.0

In addition, how can I get the exact field-of-view of my specific camera?

In the specs documents there is a tolerance of +/-3 degrees and I need the accuracy of 0.01 degree.

Thanks,

Yuval

[MartyG]

'Deprojection' is used to convert 2D pixel coordinates to 3D world coordinates, using the instruction rs2_deproject_pixel_to_point

On the 400 Series cameras, you can get the field of view with the following script code. You may need to adjust it for Python.

rs2::pipeline pipe;

rs2::pipeline_profile selection = pipe.start();

auto depth_stream = selection.get_stream(RS2_STREAM_DEPTH)

.as();

auto i = depth_stream.get_intrinsics();

float fov[2]; // X, Y fov

rs2_fov(&i, fov);

[YLitv]

Hi MartyG,

Thank you for your fast response!

I will try it out and let you know.

Yuval

[YLitv]

Hi MartyG,

The function 'get_intrinsics' doesn't exist in Python as a function of 'stream_profile'.

I have searched the web for examples and haven't found any (also regarding 'rs2_deproject_pixel_to_point').

There's a class called 'intrinsics' but I haven't figured out how to use it.

Could you please address me to some documentation or send a python code examples for using 'get_intrinsics' and 'rs2_deproject_pixel_to_point'?

Thanks!

[MartyG]

The documentation for the Python wrapper is limited. I got asked for a deproject example last week by someone else and was unable to find one. The best I can do is refer you to the Python sample programs and hope you can find the reference about Python instructions that you need in those programs.

https://github.com/IntelRealSense/librealsense/tree/master/wrappers/python/examples librealsense/wrappers/python/examples at master · IntelRealSense/librealsense · GitHub

[YLitv]

I have already read all the scripts in the link you sent and unfortunately none of them was related to my questions.

Is there anyone else you can ask?

Maybe you could connect me to the person that created the python wrapper?

[MartyG]

I do not work for Intel and so do not have any direct contact with developers. For extremely technical RealSense questions, you may get an answer if you go to the Librealsense forum, where a guy called Dorodnic posts. He is one of the Libreslsense development leaders and works with the deepest details of the cameras every day. Click the ''New Issue' button on the page below.

https://github.com/IntelRealSense/librealsense/issues/ https://github.com/IntelRealSense/librealsense/issues/

[YLitv]

I'll do as you've suggested.

Thanks a lot!

[YLitv]

Hi MartyG,

I opened an issue on github as you recommended and got answers to most of my questions, including how to use rs2_deproject_pixel_to_point.

You can have a look - https://github.com/IntelRealSense/librealsense/issues/1904 Transforming pixel from a depth image to world coordinates · Issue # 1904 · IntelRealSense/librealsense · GitHub

Thanks for all your help!

[MartyG]

Awesome, thanks for letting us know! I'm really glad they were able to help. Thanks so much for sharing the link so that others can benefit from it.