pixel depth array

[leonid_b_1]

hello,

I am using F200 camera and C# sdk ,libpxcclr.cs dll .I would like to stream in depth mode and get for every sample the depth of each pixel in my image.

           PXCMSenseManager sm = PXCMSenseManager.CreateInstance();

            // Select the color stream
            sm.EnableStream(PXCMCapture.StreamType.STREAM_TYPE_DEPTH, 640, 480);

            // Initialize and Stream samples
            sm.Init();

The code above is my preperation before start streaming. After that i get stack, How can i access to the pixel array of each frame and get the depth of each pixel.

Thanks for any help, Leonid

[jb455 · Accepted solution]

You first need to acquire a frame, using 

pSenseManager.AcquireFrame(true);

Then get the depth image:

PXCMCapture.Sample photo = pSenseManager.QuerySample();
PXCMImage depth = photo.depth;

Then get the depth data:

PXCMImage.ImageData ddata;
depth.AcquireAccess(PXCMImage.Access.ACCESS_READ, PXCMImage.PixelFormat.PIXEL_FORMAT_DEPTH, out ddata);
var dwidth = depth.info.width;
var dheight = depth.info.height;
var depthPixels = ddata.ToShortArray(0, dwidth * dheight);
depth.ReleaseAccess(ddata);
depth.Dispose()
pSenseManager.ReleaseFrame();

depthPixels is now an array containing the depth (in mm) of each pixel in the depth image. If you want to get the depth for specific points in the colour image, you'll first have to project the pixels as the two images don't align.

[leonid_b_1]

thanks James. B

When i am on STREAM_TYPE_DEPTH so sm.init return PXCM_STATUS_ITEM_UNAVAILABLE  but when i an on color mode so its work fine. What can be the resaon for it?

[iswarmrob]

PXCSenseManager *sm = PXCSenseManager::CreateInstance(); sm->EnableStream(PXCCapture::STREAM_TYPE_COLOR, 320, 240,30); sm->Init(); if (sm->AcquireFrame(true) < PXC_STATUS_NO_ERROR) { wprintf(L"没能正常打开！\n"); return 0; } PXCCapture::Sample *sample = nullptr; PXCImage *dpic = nullptr; PXCImage::ImageData dpData; // PXCImage::ImageData clData; PXCImage::ImageInfo info; while (sm->AcquireFrame(true) >= PXC_STATUS_NO_ERROR) { sample = sm->QuerySample(); dpic = sample->color; if (dpic->AcquireAccess(PXCImage::ACCESS_READ,PXCImage::PIXEL_FORMAT_RGB32, &dpData)QueryInfo(); wprintf(L"图像的深度值\n"); I failed to get the RGB values. planes[0] includes meaningless characters and planes[1~3] is null when debugging in VS 2015. Please refer tha attached screen cut. Thanks a lot.

[jb455]

leonid b. wrote:

thanks James. B

When i am on STREAM_TYPE_DEPTH so sm.init return PXCM_STATUS_ITEM_UNAVAILABLE  but when i an on color mode so its work fine. What can be the resaon for it?

Do you get an error code when you enable the stream? ie, for:

var result = pSenseManager.EnableStream(PXCMCapture.StreamType.STREAM_TYPE_DEPTH, 640,480);

What is the value of result? Make sure the camera you're using supports the resolution you're asking for.

If that's not it, I'm not sure. I only ever use the depth and colour streams together so maybe it doesn't like doing just one of them?

[jb455]

iswarmrob wrote:

PXCSenseManager *sm = PXCSenseManager::CreateInstance();

sm->EnableStream(PXCCapture::STREAM_TYPE_COLOR, 320, 240,30);
sm->Init();
if (sm->AcquireFrame(true) < PXC_STATUS_NO_ERROR) {
wprintf(L"没能正常打开！\n");
return 0;

}

PXCCapture::Sample *sample = nullptr;
PXCImage *dpic = nullptr;
PXCImage::ImageData dpData;
// PXCImage::ImageData clData;
PXCImage::ImageInfo info;

while (sm->AcquireFrame(true) >= PXC_STATUS_NO_ERROR) {
sample = sm->QuerySample();
dpic = sample->color;
if (dpic->AcquireAccess(PXCImage::ACCESS_READ,PXCImage::PIXEL_FORMAT_RGB32, &dpData)QueryInfo();
wprintf(L"图像的深度值\n");

I failed to get the RGB values. planes[0] includes meaningless characters and planes[1~3] is null when debugging in VS 2015.

Please refer tha attached screen cut. Thanks a lot.

What do you want to do with the image data? If you want to send it to your UI to display, dpData.ToBitmap(0,dpic.info.width, dpic.info.height) will allow you to do that (at least, it does in C#). If you just want to access the raw RGB data, I haven't tried doing it myself but  dpData,ToShortArray(0, dpic.info.width * dpic.info.height *3) might give you what you want.

[kfind1]

It looks like iswarmrob is trying to use colour data as depth data. There is a contradiction in what the user said (intends to do), and what the code does. They are using a Colour stream, and getting RGB data, but then expect to map it to a single channel Depth image?

[jb455]

There are two different questions in this thread: Leonid asked the original question about the depth data, then iswarmrob jacked the thread with their question about the RGB data.

[leonid_b_1]

Thanks for everything.

The crash problem that I had was resolved by installing additional depth manager driver.

I had another problem I will be glad if someone will take a look:

https://software.intel.com/en-us/forums/realsense/topic/619717

Best Regards,

Leonid