How to get the relationship between color image and depth image?

[linhuan_h_]

Hi,

Dear excellent realsense camera developers. So far, I have had my realsense cameras for a month. I learned much about this technology, but I still have some questions:

      1. Because the IR and color cameras are in the different places, the fields of view of them are different. Are there any ways to find the relationship between color image and depth image? Like human eyes, how to get the common region between right and left eye?

      2. There are two types of depth data, 16-bit unsigned integer and 32-bit floating point, and their value precision are the same (in mm). I'm confused about this setting. Empirically, the floating data is more accurate than unsigned integer.

      3. Do anyone know the relationship between precision and depth of these three 3D cameras. I got the curves between resolution and depth of  F200 and R200 from the book "Time-of-Flight and Structured Light Depth Cameras" . But I doubt the results.

      4. Could anyone introduce some solution of filtering the depth stream? I always get the depth image with rough boundary, I tried many way to get a high quality image, but failded.

Any suggestions would be appreciated.

[jb455]

1. Search the docs and/or this forum for "projection". There is lots of information out there on how to do various things with the colour and depth images.

2. Last time I checked, the float was the same as the int but with ".0". Maybe that's a bug (and maybe it's been fixed recently). If it has been fixed, I guess the int will just be the float rounded.

3. Not sure what you mean by this. How the precision changes with distance from the camera?

4. The depth image will normally be quite rough as the depth camera can only usually see some parts of the scene (other parts may be too close or far away or not reflect IR light etc). You can try projection.CreateColourImageMappedToDepth or Depth to Colour, maybe one of those will create an image you like.

[linhuan_h_]

James B. wrote:

1. Search the docs and/or this forum for "projection". There is lots of information out there on how to do various things with the colour and depth images.

2. Last time I checked, the float was the same as the int but with ".0". Maybe that's a bug (and maybe it's been fixed recently). If it has been fixed, I guess the int will just be the float rounded.

3. Not sure what you mean by this. How the precision changes with distance from the camera?

4. The depth image will normally be quite rough as the depth camera can only usually see some parts of the scene (other parts may be too close or far away or not reflect IR light etc). You can try projection.CreateColourImageMappedToDepth or Depth to Colour, maybe one of those will create an image you like.

Thanks, James. Your answers are really good.

I'm sorry for my poor English. what I want to ask in question 3 is accutually what you said.

For question 4, there is no projection.CreateColourImageMappedToDepth in the 2016 R2 SDK. And I tried the  PXCProjection::CreateColorImageMappedToDepth and  PXCProjection::CreateDepthImageMappedToColor. Although I got the same region between color and depth image, I still could not understand the theory. What's more, the depth image created by PXCProjection::CreateDepthImageMappedToColor was with rough boundary, even worse than the original depth image. Are there any ways to filter the depth stream, so that I can get an image with clear boundary?