Find depth of mapped point

[tony2]

Hello!

In C++, I used MapColorToDepth to map color coordinates to depth coordinates (returns the xy-coordinates), but I want to get the depth of those mapped points (z-coordinate). 

I was looking through the manual, but could not find a way to get the depth (z-coordinate) of a point. There must be a way to get the depth of any pixel from the depth image (i.e: from PXCImage *image = sample->depth;). Is there any function I could use? If yes, which one, and how?

Thanks in advance! 

[samontab]

Hi Tony,

You can use the depth channel to get the XYZ coordinates of each point. You can then just use the Z component or get the norm of that vector to get the distance from the center of the camera to that point in space, depending on what you need.

To get the XYZ coordinates in mm, you can just use QueryVertices from pxcprojection with your depth image:

    /** 
        @brief Retrieve the vertices for the specific depth image. The vertices is a PXCPoint3DF32 array of depth 
        size width*height. The world coordiantes units are in mm.
        @param[in]  depth        The depth image instance.
        @param[out] vertices     The 3D vertices in World coordinates, to be returned.
        @return PXC_STATUS_NO_ERROR Successful execution.
    */ 
    virtual pxcStatus PXCAPI QueryVertices(PXCImage *depth, PXCPoint3DF32 *vertices)=0;

[tony2]

Thank you Samontab!

I use QueryVertices, as you said, and I get back an array of PXCPoint3DF32 of size width*height. This array contains the depth values of each pixel. 

Now I want to get the depth value at a certain x,y - coordinates point from the depth image. Is there a way to do that without iterating over the whole width*height array? What would be the best way?

Thanks

[samontab]

You should be able to get the data for a specific point by directly accessing the array at that location. Something like:

array[y * width + x]

[Dagan_E_Intel]

You don't need to get all the vertices, you can get a pointer to the depth image data.
I haven't tried to compile this code, but it should be  something like this:

PXCImage::ImageData data;

if (image->AcquireAccess(PXCImage::ACCESS_READ,PXCImage::PIXEL_FORMAT_DEPTH, &data)>=PXC_STATUS_NO_ERROR) 

{          
            unsigned short* depthBuffer = (unsigned short*)data.planes[0];
           // now you can access the depth data - don't forget to ReleaseAccess!!!!

}

[tony2]

Thank you samontab and DAGANE E. for your answers. They both helped. 

I tried solving the problem the way samontab said, but the values I got were quite erratic, I am pretty sure due to my inexperience to work with pointers... 

Taking DAGANE's advice, I managed to get a nice and clean solution, but without using pointers:

PXCImage::ImageData depthData;
if(sample->depth->AcquireAccess(PXCImage::ACCESS_READ, PXCImage::PIXEL_FORMAT_DEPTH, &depthData) >= PXC_STATUS_NO_ERROR)
{
	// depth image height and width: 640 x 480
	unsigned short depthValues[640*480];
	memcpy(depthValues,depthData.planes[0], sizeof(unsigned short)*640*480);
	// Necessary to release the access:
	sample->depth->ReleaseAccess(&depthData);
}

// use the depthValues array

Thank you both and Happy New Year!

[Colleen_C_Intel]

BTW your comment on multiple doc sets is something that has been escalated and will hopefully both of those will be replaced with just the newest when the next release is posted.

[Lance_R_]

Colleen, 

If you're talking about the existence of multiple RSSDK reference guide documentation sources online, cool.  The comment I made was here, actually.

Lance R. wrote:

Thanks Marty, I'll have to read that carefully.

On a side note, I noticed you've linked to the v1.1 version of the documentation.  The most recent version of the SDK is the 2014GOLD.  Right off the bat when I began searching online for information, I began to realize that there are multiple (and deceptively similar) iterations of the SDK reference guide.  The "Perceptual Computing" reference guide is out of date and definitely not the one to use.  Between v1.1 and 2014GOLD, I'm not sure what the differences are in their entirety.  However, the most correct link at the present time would be here:

https://software.intel.com/sites/landingpage/realsense/camera-sdk/2014gold/documentation/html/index.html?queryuvmap_pxcprojection.html

No worries.  Thanks for your help thus far!

[Lance_R_]

tony wrote:

...

PXCImage::ImageData depthData;
if(sample->depth->AcquireAccess(PXCImage::ACCESS_READ, PXCImage::PIXEL_FORMAT_DEPTH, &depthData) >= PXC_STATUS_NO_ERROR)
{
	// depth image height and width: 640 x 480
	unsigned short depthValues[640*480];
	memcpy(depthValues,depthData.planes[0], sizeof(unsigned short)*640*480);
	// Necessary to release the access:
	sample->depth->ReleaseAccess(&depthData);
}

// use the depthValues array

Thank you both and Happy New Year!

So... coming back to this forum here, I just wanted to ask a quick question about the above snippet of code here.  Of what complex type is the mystery "sample" object?  There is no "depth" member in a PXCImage object.  If I search for "depth" in the SDK reference guide, I just get lost.

"AcquireAccess(...)" appears to be a member function of PXCImage, or is this perhaps an older, more hierarchy-based arrangement of the depth data that is no longer supported, or am I missing something??

I think "sample" represents a PXCSession.  But if that's the case, I'm still not sure exactly what I'd initialize "depth" to... like, how do I set the "sample" session to make "depth" part of its session?

[Lance_R_]

Ok, this might be riddled with errors, but this is where I am in figuring out how I can pull depth in a way that (hopefully) makes sense to me.  Like, I don't know what the "Projection" class does exactly, Marty.  It seems to me like if I wanted depth from the image, I'd be wanting to get that data from the PXCImage class per Update in Unity from the camera.  Please note this is not useful, working code here.  Just what I'm playing around with to maybe, perhaps, kind of figure out how this works.

using UnityEngine;
using System.Collections;

public class TopographicDepthMapping : BaseAction {

    PXCMImage.ImageInfo iinfo=new PXCMImage.ImageInfo() {
        width=640, height=480, format=PXCMImage.ColorFormat.COLOR_FORMAT_RGB24
    };  // is this the right format???  I want depth at this point, don't I?
    
    PXCMSession session = PXCMSession.CreateInstance();
    
    PXCMImage depth = session.CreateImage(iinfo);
    //PXCMImage::ImageData depthData;  // <-- Not sure how to access ImageData directly in C# at this point
	// Unity's Update method, called on every Update
	void Update () {
        if (depth.AcquireAccess(PXCMImage.Access.ACCESS_READ, PXCMImage.PixelFormat.PIXEL_FORMAT_DEPTH, out depthData) >= pxcmStatus.PXCM_STATUS_NO_ERROR)
        //if(sample->depth->AcquireAccess(PXCImage::ACCESS_READ, PXCImage::PIXEL_FORMAT_DEPTH, &depthData) >= PXC_STATUS_NO_ERROR)
        {
            // depth image height and width: 640 x 480
            System.UInt16[] depthValues = new System.UInt16[640*480];  // not sure what data type to use??

            //memcpy(depthValues,depthData.planes[0], sizeof(int)*640*480);
            // Copy the array to unmanaged memory.
            //Marshal.Copy(pnt, managedArray2, 0, managedArray.Length);
            System.Runtime.InteropServices.Marshal.Copy(depthValues, depthData.planes[0], 0, sizeof(int)(640*480));

            // Necessary to release the access:
            depth.ReleaseAccess(out depthData);
            //sample->depth->ReleaseAccess(&depthData);
        }
        
        // use the depthValues array
        // depthData.Dispose(); // ?
        depth.Dispose();
        session.Dispose();
	}
}

[Suresh_Kumar_B_]

I tried using this Code but when i am trying to print depthValues it prints only "0".. may i know y?

tony wrote:

Thank you samontab and DAGANE E. for your answers. They both helped. 

I tried solving the problem the way samontab said, but the values I got were quite erratic, I am pretty sure due to my inexperience to work with pointers... 

Taking DAGANE's advice, I managed to get a nice and clean solution, but without using pointers:

PXCImage::ImageData depthData;
if(sample->depth->AcquireAccess(PXCImage::ACCESS_READ, PXCImage::PIXEL_FORMAT_DEPTH, &depthData) >= PXC_STATUS_NO_ERROR)
{
	// depth image height and width: 640 x 480
	unsigned short depthValues[640*480];
	memcpy(depthValues,depthData.planes[0], sizeof(unsigned short)*640*480);
	// Necessary to release the access:
	sample->depth->ReleaseAccess(&depthData);
}

// use the depthValues array

Thank you both and Happy New Year!