For one color pixel how to get its 3D coordinate in the space defined by color camera?

[Yixun_L_]

Hi,

For one color pixel, how to get its 3D coordinate? 

In RealSense SDK, the 3D coordinate is referred to the depth camera.

But I want to find 3D coordinate in the color camera space.

How to do this?

Thanks a lot.

YL

[samontab]

You need to project one into the other to match them.

Have a look here:

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?manuals_coordinates_mapping.html

[Yixun_L_]

Thanks.

I checked the Projection interface, which can map a pixel in the color image to the 3D coordinate.

But I think it is the 3D coordinate in the depth camera coordinate system rather than in the color camera coordinate system.

I think my original question can be:

Does the SDK provide the transform between the depth and the color camera?

Thanks.

[samontab]

Of course, have a look here:

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?member_functions_pxcprojection.html

[Henning_J_]

@Yixun L.:

You're right. You can use PXCProjection::ProjectColorToCamera() to map color pixels to 3D coordinates in the world coordinate system (in millimeters). But, as you said, the world coordinate system "coincides with the depth camera coordinate system".

To transform that into a coordinate system based on the color camera, you need to get the Calibration interface from the Projection, and then use PXCCalibration::QueryStreamProjectionParameters() to get the PXCCalibration::StreamTransform struct of the color camera. The rotation is most likely the identity matrix, but the translation denotes distance (in millimeter) of the color camera to the depth camera (and therefore to the world coordinate system).

So just apply the translation to your 3D points and you should have the points in a coordinate system based on the color camera.