topic Re: pixels mismatch in Items with no label

pixels mismatch

fwang22 — Sun, 30 Dec 2018 21:00:36 GMT

When I deal with the color images and depth images, I find the pixels are not consistence in the color and depth image. ,there is an offset between the color images and the corresponding depth images. So what should I do?

Re: pixels mismatch

MartyG — Sun, 30 Dec 2018 21:18:02 GMT

The physical RGB and IR imager sensors in the camera are in different positions. So I would recommend performing an Align operation.

In the link below is a tutorial for aligning color with depth.

https://github.com/IntelRealSense/librealsense/tree/master/examples/align

Re: pixels mismatch

fwang22 — Sun, 30 Dec 2018 22:12:06 GMT

I want to know if there is a way to align an image in the viewer, or how to align a color image with its corresponding depth image use code because I have obtained the color and depth images

Re: pixels mismatch

MartyG — Sun, 30 Dec 2018 23:22:52 GMT

An alternative to using the Align example program is to refer to this guide on the Align processing block:

https://github.com/IntelRealSense/librealsense/wiki/Projection-in-RealSense-SDK-2.0?language=en_US#frame-alignment

The RealSense Viewer can align color with depth in its 3D point cloud mode. I do not know of a way to do alignment in ithe Viewer's default 2D mode though. A point cloud can have color and depth aligned in 3D mode using the process below:

STEP ONE

Enable both the depth stream and the RGB stream whilst in the default 2D mode

STEP TWO

Click on the '3D' button in the top corner of the RealSense Viewer to enter 3D point cloud mode.

STEP THREE

Click on the drop-down menu called 'Texture Source' and select the 'Color' option. Do not change the 'Depth Source' menu above it.

RGB and depth should now be aligned in the point cloud.

Re: pixels mismatch

fwang22 — Sun, 30 Dec 2018 23:53:14 GMT

I've tried this before. The point cloud extracted from the.bag file is color - and depth-aligned, but in a two-dimensional image the color - and depth-aligned image is not。Is that true?

Re: pixels mismatch

MartyG — Mon, 31 Dec 2018 00:04:17 GMT

Yes, it is usually recommended that 2D alignments are done with code, using scripting such as the Align processing block that I linked to in my previous message.

An alternate method to the rs-align sample program to align color and depth is in the sample program 'grabcuts'

https://github.com/IntelRealSense/librealsense/tree/master/wrappers/opencv/grabcuts?language=en_US

Re: pixels mismatch

fwang22 — Mon, 31 Dec 2018 11:49:53 GMT

I wonder if alignment can only be done when sampling, there is no way to align the samples that have been collected.

If select the 'Color' option in the'Texture Source' , point cloud and color are pixel aligned?

Re: pixels mismatch

MartyG — Mon, 31 Dec 2018 15:27:32 GMT

It is possible to do processing on non real-time data that has already been recorded, such as .raw and .bag files or PNG images of color and depth.

Here are a couple of example discussions about doing so with images.

https://github.com/IntelRealSense/librealsense/issues/2231

https://github.com/IntelRealSense/librealsense/issues/2534

Re: pixels mismatch

fwang22 — Tue, 01 Jan 2019 12:40:26 GMT

I want to do it use matlab code, because the above discussions is so confusing for me.

Re: pixels mismatch

MartyG — Tue, 01 Jan 2019 16:20:18 GMT

If you have a Windows PC, there is a compatibility 'wrapper' for MATLAB in the RealSense SDK 2.0 that can do alignment with a function called 'align.m'.

https://github.com/IntelRealSense/librealsense/tree/master/wrappers/matlab

The discussion linked to below talks about using align.m to do alignment between color and depth images.

https://github.com/IntelRealSense/librealsense/issues/2455

Re: pixels mismatch

fwang22 — Tue, 01 Jan 2019 16:27:26 GMT

I have seen the answer :

https://github.com/IntelRealSense/librealsense/issues/2455#issuecomment-439939852

but I think it aligns the images in real time.

Re: pixels mismatch

MartyG — Tue, 01 Jan 2019 16:43:33 GMT

Another way to do alignment may be with a MATLAB product called the Image Processing Toolbox. It is not inexpensive, but it does have a free trial version.

https://uk.mathworks.com/products/image.html

Re: pixels mismatch

fwang22 — Tue, 01 Jan 2019 17:09:08 GMT

I don't know if the above method works for aligning color and depth images,but I will try it . Thank you very much for your timely reply, which is very important to me.

Re: pixels mismatch

fwang22 — Wed, 02 Jan 2019 12:05:06 GMT

I tried this method to align color images and depth images, but the results were disappointing.

Another problem, when I extract the point cloud using convert, there are always errors in CMAKE.

I have tried some methods, but it still doesn't work. Can you tell me how to solve it? Thank you.

Re: pixels mismatch

fwang22 — Wed, 02 Jan 2019 12:06:12 GMT

I tried this method to align color images and depth images, but the results were disappointing.

Another problem, when I extract the point cloud using convert, there are always errors in CMAKE.

I have tried some methods, but it still doesn't work. Can you tell me how to solve it? Thank you.

Re: pixels mismatch

MartyG — Wed, 02 Jan 2019 15:17:40 GMT

I tracked down a further discussion on aligning non-live depth and color images.

https://github.com/IntelRealSense/librealsense/issues/1274

Re: pixels mismatch

fwang22 — Wed, 02 Jan 2019 17:41:42 GMT

I want to use ‘rs-convert’ to get the point cloud, because I used to extract depth also for point clouds. But when I extract the point cloud, it's always terminated.

Re: pixels mismatch

MartyG — Wed, 02 Jan 2019 18:24:38 GMT

The rs-convert tool is for converting bag files to other formats, such ans PNG (image) and ply (point cloud). Are you trying to convert a bag file, please?

Re: pixels mismatch

fwang22 — Wed, 02 Jan 2019 19:20:07 GMT

Yes, I want to convert .bag file into .ply file,but the 'rs-convert' always stops as above.

Re: pixels mismatch

MartyG — Wed, 02 Jan 2019 20:17:03 GMT

If you need the point cloud to have color and depth aligned, doing a bag to ply conversion of 2D data is probably not the best way to do it. This is because 3D point clouds need to take the "extrinsic" values of the camera into account when doing alignment, otherwise the results will be inaccurate.

I will refer this case directly to an Intel support member in the hope that they can provide a quick answer.

@Jesus G.