Realsense ImageData to OpenCV Mat

[Marios_B_]

Hi there!

Can someone tell me how I can get the realsense imagedata into a cv::Mat?

I found some stuff in the forum,but I was confused with the planes/pitches.Any links/sample code for conversion from realsense to mat formats?

[samontab · Accepted solution]

It depends on what type of data you want to get, and in what format.

For example, to get a color image, you could do something like this: (it can be done more efficiently, and you should error check on each step, but this should be easy to understand)

Let's say you have a color image:

PXCImage *color

You can create a cv::Mat color image based on it:

//First, create an ImageData object:

PXCImage::ImageData * data = new PXCImage::ImageData;

//Set the format that you want to use:
data->format = PXCImage::PIXEL_FORMAT_RGB32; // For example 4 bytes in this case (8 * 4 = 32)

//Create a cv::Mat black canvas based on the size of the color image

int width = color->QueryInfo().width;
int height = color->QueryInfo().height;
cv::Mat frameColor = cv::Mat::zeros(height, width, CV_8UC3);

//Now get access to the memory of the image:

color->AcquireAccess(PXCImage::ACCESS_READ, data->format, data);

//Now we just need to paint the canvas with each pixel from the image data

 pxcBYTE *cpixels=data->planes[0];
 
int x = 0, y = 0;

        for(int i=0; i < width * height; i++)
        {
            //BGRA
            //Read the first 3 bytes of each 4 bytes, and ignore the last one
            pxcBYTE bvalue = cpixels[i * 4 + 0];
            pxcBYTE gvalue = cpixels[i * 4 + 1];
            pxcBYTE rvalue = cpixels[i * 4 + 2];
            unsigned int bvalueUchar = static_cast<unsigned int>(bvalue);
            unsigned int gvalueUchar = static_cast<unsigned int>(gvalue);
            unsigned int rvalueUchar = static_cast<unsigned int>(rvalue);
 
            frameColor.at<cv::Vec3b>(y, x)[0] = (bvalueUchar);
            frameColor.at<cv::Vec3b>(y, x)[1] = (gvalueUchar);
            frameColor.at<cv::Vec3b>(y, x)[2] = (rvalueUchar);
            //Advance, one pixel at a time (can be done better of course!)
            x += 1;
            if(x >= width)
            {
                x = 0;
                y += 1;
            }
        }
        //At the end, release all the used memory
        color->ReleaseAccess(data);
        delete data;

[Marios_B_]

Thank you for your reply.

I now understood the planes thing.However what exactly is the pitch?

Also,what happens if I also want the depth pixels?

Same thing exactly but with only 1 value per pixel?

[samontab]

Hi Marios,

I recently made a video tutorial explaining how to get the depth, IR, and colour streams into OpenCV using a single and a multi threaded approach. You might find this useful:

https://software.intel.com/en-us/forums/realsense/topic/595033