RealSense SDK for Depth Information

[Shrutish_R_]

Hello,

Is there an updated SDK for getting the depth information (distance) of each pixels in an image ? I am using Intel RealSense SR300. Please Help !

[Shrutish_R_]

I am using C# sdk .I would like to stream in depth mode and get for every sample the depth of each pixel in my image.

[jb455 · Accepted solution]

Assuming you already have a depth image, this code gives you an array of depth values (rounded to the nearest mm) the size of the depth image:

dImage.AcquireAccess(PXCMImage.Access.ACCESS_READ, PXCMImage.PixelFormat.PIXEL_FORMAT_DEPTH, out ddata);
dwidth = dImage.info.width;
var dheight = dImage.info.height;
dpitch = dwidth; //(int)(mdata.pitches[0] / 2.0);
Get the depth values:
dPixels = ddata.ToShortArray(0, dwidth * dheight);
dImage.ReleaseAccess(mdata);
dImage.Dispose();

So if you want the depth for a specific pixel, say (u, v), it'll be dPixels[v * dpitch + u]

[Shrutish_R_]

Hello,

I am actually new for the C# . All I want is to get the depth information of each pixels for further processing. Which is the best Framework Sample sources in C# (Provided in the SDK) can be used for getting the data.? Also it would of great help if the code was provided.

Thank you in advance.

[jb455]

The Raw Streams sample is a good place to start if you want to learn how it all works.

Be aware, if you want to get depth data for points in the colour image, you'll have to map between the colour and depth images as the two cameras point to different locations and have different fields of view etc. Search the docs and this forum for 'projection' and there's information on most use cases.

[Shrutish_R_]

Thank you James .But I tried retrieving the data. Was not able to do so . It would be of a great help if you could provide the code for getting the depth data(in mm) of each pixels and storing it in an array using the Raw Streams Sample .

[jb455]

What have you tried doing?

If you look at the Raw Streams code, in the main camera streaming loop there's sample = sm.QuerySample(). If you have enabled a depth stream, the depth image (called dImage in my code above) is contained within this sample object (ie, sample.depth).

[Shrutish_R_]

Is there a definition or a method for the "AcquireAccess "  in the SR300 SDK ?

[jb455]

Have you tried looking in the documentation?

[Shrutish_R_]

Hai James,

I tried doing this . Is it possible to get the depth data of each pixels now ?
thank you in advance.







 

 private void button1_Click(object sender, EventArgs e)
        {

            PXCMPoint3DF32[] mDepthPoints;
            short[] mDepthBuffer;
            mDepthBuffer = new short[640 * 480];
            mDepthPoints = new PXCMPoint3DF32[640 * 480];

            // Create a SenseManager instance

            PXCMSenseManager sm = PXCMSenseManager.CreateInstance();

            // Select the color and depth streams
            sm.EnableStream(PXCMCapture.StreamType.STREAM_TYPE_DEPTH, 320, 240, 30);

            // Initialize and Stream Samples

            sm.Init();

            for (; ; )
            {

                // This function blocks until both samples are ready

                if (sm.AcquireFrame(true).IsError()) break;

                // retrieve the samples

                PXCMCapture.Sample sample = sm.QuerySample();

                if (sample.depth != null)
                {

                    PXCMImage.ImageData ddata = new PXCMImage.ImageData();
                    sample.depth.AcquireAccess(PXCMImage.Access.ACCESS_READ, PXCMImage.PixelFormat.PIXEL_FORMAT_DEPTH, out ddata);
                    mDepthBuffer = ddata.ToShortArray(0, mDepthBuffer);

                    int dID = 0;
                    for (int dy = 0; dy < 480; ++dy)
                    {
                        for (int dx = 0; dx < 640; dx++)
                        {
                            PXCMPoint3DF32 dPoint = new PXCMPoint3DF32();
                            dPoint.x = dx;
                            dPoint.y = dy;
                            dPoint.z = (float)mDepthBuffer[dID];
                            mDepthPoints[dID] = dPoint;
                            ++dID;

                        }
                    }
         
                    sample.depth.ReleaseAccess(ddata);
                }
               // get next samples

                sm.ReleaseFrame();

            }
            //release the frame
            sm.Dispose();

        }

[jb455]

That looks about right - mDepthBuffer should contain the depth of each pixel in the depth image. Is that not what you wanted?

[Shrutish_R_]

Hi James, Thank you.
, Yes, This is what I was looking for . But Since I am using the loop , getting the data out is very slow ! any other ideas for this problem ??|
 

[jb455]

Do you need all the depth pixels every frame? Could you do it every 10th frame or something instead? Also your loop to convert the depth points to PXCMPoints is kinda pointless; if you need specific pixels you can use mDepthBuffer[v * 640 * u] to get the depth for (u,v) in the depth image.

[Shrutish_R_]

Actually I need all the depth pixels of Just 1 frame.

[Shrutish_R_]

Hi,

Can anyone help me with this ? The below is the code for getting the depth data of each pixels. The depth data is available in  " depthPixels "

However, printing the values to the console consumes lot of time. Is there any possible way I can do it ?
Thank you

 private void button1_Click(object sender, EventArgs e)
        {
            // Create a SenseManager instance

            PXCMSenseManager sm = PXCMSenseManager.CreateInstance();

            // Select the color and depth stream
            sm.EnableStream(PXCMCapture.StreamType.STREAM_TYPE_DEPTH, 640, 480, 30);

            // Initialize and Stream Samples

            sm.Init();

            for (; ; )
            {

                // This function blocks until both samples are ready
                sm.AcquireFrame(true);
                if (sm.AcquireFrame(true).IsError()) break;

                // retrieve the samples

                PXCMCapture.Sample photo = sm.QuerySample();
                PXCMImage depth = photo.depth;

                PXCMImage.ImageData ddata;
                depth.AcquireAccess(PXCMImage.Access.ACCESS_READ, PXCMImage.PixelFormat.PIXEL_FORMAT_DEPTH, out ddata);
                var dwidth = depth.info.width;
                var dheight = depth.info.height;
                var depthPixels = ddata.ToShortArray(0, dwidth * dheight);
                depth.ReleaseAccess(ddata);
                depth.Dispose();
                sm.ReleaseFrame();

// getting each element from the array
                foreach (int i in depthPixels)
                {
                    System.Console.Write(i);
                }

            }

[Alex_P_5]

 What have you tried doing?

 If you look at the Raw Streams code, in the main camera streaming loop there's sample = sm.QuerySample(). If you have enabled a depth stream, the  depth image (called dImage in my code above) is contained within this sample object (ie, sample.depth).

[jb455]

You said in your previous reply that you only need the depth data in one frame. Do you have a button or something in your UI which sets a bool to tell the camera that "this is the frame"? I'd suggest doing something like that and putting everything from "PXCMImage.ImageData ddata;" in an if statement based on the bool set by the UI.

Also, why do you want to write every pixel to the console? Will that actually be used by anyone? I don't know what the purpose of your app is but surely having 307,200 numbers written in the console isn't much use?

[Niyonsaba_E_]

Hello. I want just ask the accuracy of Intel RealSense camera F200 or SR300. With the same test environment, if you execute your code 2 times, you can get same depths of all pixels in the first time and second time. I tried for other depth camera as kinect but i got different distances for the same pixel and in the same environment. How about RealSense camera?

[Shrutish_R_]

@Niyonsaba

Hello,

I am using the RealSense SR300. Well, this camera gave me the "SAME" distance for the same pixel in the same environment when I executed the code  10 times ! SR300 has a very good accuracy.

[Shrutish_R_]

@ James

Hello,

Yes. I have a button in my UI. I want the depth information of each pixels to be written to an excel sheet as this data is used by someone for further processing. So, before storing these data to an excel , I just wanted to print these data to the console for my reference. 

Now, It works perfectly fine. Thank you for the guidance.

[Niyonsaba_E_]

@James

Hello.

I am using R 200 camera with WPF. All depth values equal to zero. I am wondering if there is another way to get an array of depth data with R200 camera. I want to get depth values for oneframe.