resize COLOR, DEPTH & IR

[NB3]

Hi,

I'm currently trying the SDK R2 (in java and c++) and I have some issues concerning the resize of streams.

The color stream is in 1920x1080, depth and IR are in 640x480.

I'd like to scale the 3 streams so they are at exactly the same size and stackable (currently depth & IR are ok, obviously)

Ideally, they should be at the same size as color stream (1920x1080) so I can also add landmark points.

I tried QueryUVMap, QueryInvUVMap, but in any case I loose information : if I put color to depth size, there are only the color pixels corresponding to depth. if I put depth to color size, it's ok but I can't resize IR to color size.

Is there any way to achieve having the three streams at the same size ?

I think I'm actually very close, the main problematic stream is IR that can't be correctly resized..

Thanks in advance for your help !

[MartyG]

It's normally best practice to have all displayed streams be set to the same resolution. Depth and IR do not normally support resolutions greater than 640x480 though, as 1920x1080 is only available in the color mode.

Having said that, you might want to try CreateDepthImageMappedToColor, which "creates a depth image that is spatially aligned with, and the same size as the color image".

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?createdepthimagemappedtocolor_pxcprojection.html Intel® RealSense™ SDK 2016 R2 Documentation

Here's a link to someone who tried this method to mix color and depth into the same size.

https://putuyuwono.wordpress.com/2016/03/17/realsense-sdk-calibrate-color-and-depth-stream/ [RealSense SDK] Calibrate Color and Depth Stream | Putu Yuwono Kusmawan

I don't know of an equivalent instruction to map IR to the 1920x1080 color resolution size though, which was the main problem that you were trying to solve. I did not find any cases of IR having been resized beyond 640x480 during my research into your question.

[NB3]

Hi,

Thanks for your answer.

I need a bigger color stream size for better quality, so I can avoid having a webcam only for the purpose of getting a 1080p image (actually it will be displayed on a 42" screen in portrait mode)

I already tried CreateDepthImageMappedToColor, and it's working nicely.

I still have the problem with IR...

(I should have been clearer and focus on the IR image that I can't resize )

Perhaps I could try converting the IR PXCMImage to be the same format as depth so I could use CreateDepthImageMappedToColor on the IR one ?

Well, if anyone has a solution to scale the IR image to color size, I really need it !

[MartyG]

This link contains details of how someone mapped IR onto the depth image.

[NB3]

I continued investigating with some ideas and your links.

Now I'm able to have, with the size of the RGB stream (1920x1080) : RGB, depth, IR masked by depth.

This means that in the scaled IR image (scaled IR size = color size), only pixels that matches depth are visible (because I use QueryInvUVMap, don't know if I'm clear enough)

So I still miss a lot of information of the IR stream, because of QueryInvUVMap, but that's still a big step !

[MartyG]

I'm glad you were able to make progress!

The same user as that previous link posted another message the next day with further information that may be of interest to you.