topic Right, so have you done as I in Software Archive

Export coordinates F200 camera

rayan_g_ — Mon, 26 Oct 2015 03:30:32 GMT

Hi!

I am new with RealSense camera F200.

I am trying export and save the coordinates as a matrix (x,y,z) from any RealSense code for example Hand Console C++ code.

How can I do that?

Regards,

Hi,

samontab — Mon, 26 Oct 2015 06:04:40 GMT

Hi,

If you have a depth image, you can get X, Y, Z (in mm), with this:

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?queryvertices_pxcprojection.html

Thank you for your answer Sir

rayan_g_ — Tue, 03 Nov 2015 04:13:10 GMT

Thank you for your answer Sir

I tried to use this function, but I couldn't get the (x,y,z) as output and save them as a matrix.

Could you please give me an example; because in the link above there is no any example, just the syntax of this function.

Thank you for your patient.

Why not use the software

Ariska_Hidayat — Tue, 03 Nov 2015 18:12:29 GMT

Why not use the software unity? I think it would be easier to use unity.

Hi Rayan,

jb455 — Wed, 04 Nov 2015 09:35:14 GMT

Hi Rayan,

I recently explained how to do this in another thread here: https://software.intel.com/en-us/forums/realsense/topic/595370 (in C# but it should be mostly the same)

It outputs a single-dimensional array though, not a matrix, so to get a specific (x.y) point you'll need to do vertices[y*depth.info.width + x] (where (x,y) is a depth point - if you have a colour point you'll need to map it like this: https://software.intel.com/en-us/forums/realsense/topic/596309)

James

Quote:Ariska Hidayat wrote:

rayan_g_ — Wed, 04 Nov 2015 13:33:55 GMT

Ariska Hidayat wrote:

Why not use the software unity? I think it would be easier to use unity.

I am haven't use it before, but i will try

Thank you

Quote:James N. wrote:

rayan_g_ — Wed, 04 Nov 2015 14:26:44 GMT

James N. wrote:

Hi Rayan,

I recently explained how to do this in another thread here: https://software.intel.com/en-us/forums/realsense/topic/595370 (in C# but it should be mostly the same)

It outputs a single-dimensional array though, not a matrix, so to get a specific (x.y) point you'll need to do vertices[y*depth.info.width + x] (where (x,y) is a depth point - if you have a colour point you'll need to map it like this: https://software.intel.com/en-us/forums/realsense/topic/596309)

James

Thank you James

I tried to use your code as shown below, but I always get error messages

I don't know if put the code in wrong place ? you can see it between (///try15) comment in the following code.

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Drawing;
using System.Drawing.Imaging;
using System.Linq;

namespace hands_viewer.cs
{
    class HandsRecognition
    {
        byte[] LUT;
        private MainForm form;
        private bool _disconnected = false;
        //Queue containing depth image - for synchronization purposes
        private Queue<PXCMImage> m_images;
        private const int NumberOfFramesToDelay = 3;
        private int _framesCounter = 0;
        private float _maxRange;

        public HandsRecognition(MainForm form)
        {
            m_images = new Queue<PXCMImage>();
            this.form = form;
            LUT = Enumerable.Repeat((byte)0, 256).ToArray();
            LUT[255] = 1;
        }

        /* Checking if sensor device connect or not */
        private bool DisplayDeviceConnection(bool state)
        {
            if (state)
            {
                if (!_disconnected) form.UpdateStatus("Device Disconnected");
                _disconnected = true;
            }
            else
            {
                if (_disconnected) form.UpdateStatus("Device Reconnected");
                _disconnected = false;
            }
            return _disconnected;
        }

        /* Displaying current frame gestures */
        private void DisplayGesture(PXCMHandData handAnalysis,int frameNumber)
        {

            int firedGesturesNumber = handAnalysis.QueryFiredGesturesNumber();
            string gestureStatusLeft = string.Empty;
            string gestureStatusRight = string.Empty;
            if (firedGesturesNumber == 0)
            {

                return;
            }

            for (int i = 0; i < firedGesturesNumber; i++)
            {
                PXCMHandData.GestureData gestureData;
                if (handAnalysis.QueryFiredGestureData(i, out gestureData) == pxcmStatus.PXCM_STATUS_NO_ERROR)
                {
                    PXCMHandData.IHand handData;
                    if (handAnalysis.QueryHandDataById(gestureData.handId, out handData) != pxcmStatus.PXCM_STATUS_NO_ERROR)
                        return;

                    PXCMHandData.BodySideType bodySideType = handData.QueryBodySide();
                    if (bodySideType == PXCMHandData.BodySideType.BODY_SIDE_LEFT)
                    {
                        gestureStatusLeft += "Left Hand Gesture: " + gestureData.name;
                    }
                    else if (bodySideType == PXCMHandData.BodySideType.BODY_SIDE_RIGHT)
                    {
                        gestureStatusRight += "Right Hand Gesture: " + gestureData.name;
                    }

                }

            }
            if (gestureStatusLeft == String.Empty)
                form.UpdateInfo("Frame " + frameNumber + ") " + gestureStatusRight + "\n", Color.SeaGreen);
            else
                form.UpdateInfo("Frame " + frameNumber + ") " + gestureStatusLeft + ", " + gestureStatusRight + "\n",Color.SeaGreen);

        }

        /* Displaying Depth/Mask Images - for depth image only we use a delay of NumberOfFramesToDelay to sync image with tracking */
        private unsafe void DisplayPicture(PXCMImage depth, PXCMHandData handAnalysis)
        {
            if (depth == null)
                return;

PXCMImage image = depth;

            //Mask Image
            if (form.GetLabelmapState())
            {
                Bitmap labeledBitmap = null;
                try
                {
                    int numOfHands = handAnalysis.QueryNumberOfHands();

PXCMPointI32[][] pointOuter = new PXCMPointI32[numOfHands][];
PXCMPointI32[][] pointInner = new PXCMPointI32[numOfHands][];

                    labeledBitmap = new Bitmap(image.info.width, image.info.height, PixelFormat.Format32bppRgb);
                    for (int j = 0; j < numOfHands; j++)
                    {
                        int id;
                        PXCMImage.ImageData data;

                        handAnalysis.QueryHandId(PXCMHandData.AccessOrderType.ACCESS_ORDER_BY_TIME, j, out id);
                        //Get hand by time of appearance
                        PXCMHandData.IHand handData;
                        handAnalysis.QueryHandData(PXCMHandData.AccessOrderType.ACCESS_ORDER_BY_TIME, j, out handData);
                        if (handData != null &&
                            (handData.QuerySegmentationImage(out image) >= pxcmStatus.PXCM_STATUS_NO_ERROR))
                        {
                            if (image.AcquireAccess(PXCMImage.Access.ACCESS_READ, PXCMImage.PixelFormat.PIXEL_FORMAT_Y8,
                                out data) >= pxcmStatus.PXCM_STATUS_NO_ERROR)
                            {
                                Rectangle rect = new Rectangle(0, 0, image.info.width, image.info.height);

                                BitmapData bitmapdata = labeledBitmap.LockBits(rect, ImageLockMode.ReadWrite,
                                    labeledBitmap.PixelFormat);
                                byte* numPtr = (byte*) bitmapdata.Scan0; //dst
                                byte* numPtr2 = (byte*) data.planes[0]; //row
                                int imagesize = image.info.width*image.info.height;
                                byte num2 = (byte) handData.QueryBodySide();

                                byte tmp = 0;
                                for (int i = 0; i < imagesize; i++, numPtr += 4, numPtr2++)
                                {
                                    tmp = (byte) (LUT[numPtr2[0]]*num2*100);
                                    numPtr[0] = (Byte) (tmp | numPtr[0]);
                                    numPtr[1] = (Byte) (tmp | numPtr[1]);
                                    numPtr[2] = (Byte) (tmp | numPtr[2]);
                                    numPtr[3] = 0xff;
                                }

                                labeledBitmap.UnlockBits(bitmapdata);
                                image.ReleaseAccess(data);

                            }

                            if ((form.GetContourState()))
                            {
                                int contourNumber = handData.QueryNumberOfContours();
                                if (contourNumber > 0)
                                {
                                    for (int k = 0; k < contourNumber; ++k)
                                    {
                                        PXCMHandData.IContour contour;
                                        pxcmStatus sts = handData.QueryContour(k, out contour);
                                        if (sts == pxcmStatus.PXCM_STATUS_NO_ERROR)
                                        {
                                            //int contourSize = contour.QuerySize();
                                            if (contour.IsOuter() == true)
                                                contour.QueryPoints(out pointOuter);
                                            else
                                            {
                                                contour.QueryPoints(out pointInner);
                                            }
                                        }
                                    }

}
}

                        }
                    }
                    if (labeledBitmap != null)
                    {

                        form.DisplayBitmap(labeledBitmap);
                        labeledBitmap.Dispose();
                    }
                    image.Dispose();

                    for (int i = 0; i < numOfHands; i++)
                    {
                        if (form.GetContourState())
                        {
                            if (pointOuter != null && pointOuter.Length > 0)
                                form.DisplayContour(pointOuter, i);
                            if (pointInner != null && pointInner.Length > 0)
                                form.DisplayContour(pointInner, i);
                        }

}

                }
                catch (Exception)
                {
                    if (labeledBitmap != null)
                    {
                        labeledBitmap.Dispose();
                    }
                    if (image != null)
                    {
                        image.Dispose();
                    }
                }

}//end label image

            //Depth Image
            else
            {
                //collecting 3 images inside a queue and displaying the oldest image
                PXCMImage.ImageInfo info;
                PXCMImage image2;

                info = image.QueryInfo();
                image2 = form.g_session.CreateImage(info);
                if (image2 == null) { return; }
                image2.CopyImage(image);
                m_images.Enqueue(image2);
                if (m_images.Count == NumberOfFramesToDelay)
                {
                    Bitmap depthBitmap;
                    try
                    {
                        depthBitmap = new Bitmap(image.info.width, image.info.height, PixelFormat.Format32bppRgb);
                    }
                    catch (Exception)
                    {
                        image.Dispose();
                        PXCMImage queImage = m_images.Dequeue();
                        queImage.Dispose();
                        return;
                    }

                    PXCMImage.ImageData data3;
                    PXCMImage image3 = m_images.Dequeue();
                    if (image3.AcquireAccess(PXCMImage.Access.ACCESS_READ, PXCMImage.PixelFormat.PIXEL_FORMAT_DEPTH, out data3) >= pxcmStatus.PXCM_STATUS_NO_ERROR)
                    {
                        float fMaxValue = _maxRange;
                        byte cVal;

Rectangle rect = new Rectangle(0, 0, image.info.width, image.info.height);
BitmapData bitmapdata = depthBitmap.LockBits(rect, ImageLockMode.ReadWrite, depthBitmap.PixelFormat);

                        byte* pDst = (byte*)bitmapdata.Scan0;
                        short* pSrc = (short*)data3.planes[0];
                        int size = image.info.width * image.info.height;

                        for (int i = 0; i < size; i++, pSrc++, pDst += 4)
                        {
                            cVal = (byte)((*pSrc) / fMaxValue * 255);
                            if (cVal != 0)
                                cVal = (byte)(255 - cVal);

                            pDst[0] = cVal;
                            pDst[1] = cVal;
                            pDst[2] = cVal;
                            pDst[3] = 255;
                        }
                        try
                        {
                            depthBitmap.UnlockBits(bitmapdata);
                        }
                        catch (Exception)
                        {
                            image3.ReleaseAccess(data3);
                            depthBitmap.Dispose();
                            image3.Dispose();
                            return;
                        }

                        form.DisplayBitmap(depthBitmap);
                        image3.ReleaseAccess(data3);
                    }
                    depthBitmap.Dispose();
                    image3.Dispose();
                }
            }
        }

        /* Displaying current frames hand joints */
        private void DisplayJoints(PXCMHandData handOutput, long timeStamp = 0)
        {
            if (form.GetJointsState() || form.GetSkeletonState())
            {
                //Iterate hands
                PXCMHandData.JointData[][] nodes = new PXCMHandData.JointData[][] { new PXCMHandData.JointData[0x20], new PXCMHandData.JointData[0x20] };
                int numOfHands = handOutput.QueryNumberOfHands();
                for (int i = 0; i < numOfHands; i++)
                {
                    //Get hand by time of appearence
                    //PXCMHandAnalysis.HandData handData = new PXCMHandAnalysis.HandData();
                    PXCMHandData.IHand handData;
                    if (handOutput.QueryHandData(PXCMHandData.AccessOrderType.ACCESS_ORDER_BY_TIME, i, out handData) == pxcmStatus.PXCM_STATUS_NO_ERROR)
                    {
                        if (handData != null)
                        {
                            //Iterate Joints
                            for (int j = 0; j < 0x20; j++)
                            {
                                PXCMHandData.JointData jointData;
                                handData.QueryTrackedJoint((PXCMHandData.JointType)j, out jointData);
                                nodes = jointData;

                            } // end iterating over joints
                        }
                    }
                } // end itrating over hands

                form.DisplayJoints(nodes, numOfHands);
            }
            else
            {
                form.DisplayJoints(null, 0);
            }
        }

        /* Displaying current frame alerts */
        private void DisplayAlerts(PXCMHandData handAnalysis, int frameNumber)
        {
            bool isChanged = false;
            string sAlert = "Alert: ";
            for (int i = 0; i < handAnalysis.QueryFiredAlertsNumber(); i++)
            {
                PXCMHandData.AlertData alertData;
                if (handAnalysis.QueryFiredAlertData(i, out alertData) != pxcmStatus.PXCM_STATUS_NO_ERROR)
                    continue;

                //See PXCMHandAnalysis.AlertData.AlertType for all available alerts
                switch (alertData.label)
                {
                    case PXCMHandData.AlertType.ALERT_HAND_DETECTED:
                        {

                            sAlert += "Hand Detected, ";
                            isChanged = true;
                            break;
                        }
                    case PXCMHandData.AlertType.ALERT_HAND_NOT_DETECTED:
                        {

                            sAlert += "Hand Not Detected, ";
                            isChanged = true;
                            break;
                        }
                    case PXCMHandData.AlertType.ALERT_HAND_CALIBRATED:
                        {

                            sAlert += "Hand Calibrated, ";
                            isChanged = true;
                            break;
                        }
                    case PXCMHandData.AlertType.ALERT_HAND_NOT_CALIBRATED:
                        {

                            sAlert += "Hand Not Calibrated, ";
                            isChanged = true;
                            break;
                        }
                    case PXCMHandData.AlertType.ALERT_HAND_INSIDE_BORDERS:
                        {

                            sAlert += "Hand Inside Border, ";
                            isChanged = true;
                            break;
                        }
                    case PXCMHandData.AlertType.ALERT_HAND_OUT_OF_BORDERS:
                        {

                            sAlert += "Hand Out Of Borders, ";
                            isChanged = true;
                            break;
                        }
                }
            }
            if (isChanged)
            {
                form.UpdateInfo("Frame " + frameNumber + ") " + sAlert + "\n", Color.RoyalBlue);
            }

}

        public static pxcmStatus OnNewFrame(Int32 mid, PXCMBase module, PXCMCapture.Sample sample)
        {
            return pxcmStatus.PXCM_STATUS_NO_ERROR;
        }

        /* Using PXCMSenseManager to handle data */
        public void SimplePipeline()
        {
            form.UpdateInfo(String.Empty,Color.Black);
            bool liveCamera = false;

            bool flag = true;
            PXCMSenseManager instance = null;
            _disconnected = false;
            instance = form.g_session.CreateSenseManager();
            if (instance == null)
            {
                form.UpdateStatus("Failed creating SenseManager");
                return;
            }

            PXCMCaptureManager captureManager = instance.captureManager;
            if (captureManager != null)
            {
                if (form.GetRecordState())
                {
                    captureManager.SetFileName(form.GetFileName(), true);
                    PXCMCapture.DeviceInfo info;
                    if (form.Devices.TryGetValue(form.GetCheckedDevice(), out info))
                    {
                        captureManager.FilterByDeviceInfo(info);
                    }

                }
                else if (form.GetPlaybackState())
                {
                    captureManager.SetFileName(form.GetFileName(), false);
                }
                else
                {
                    PXCMCapture.DeviceInfo info;
                    if (String.IsNullOrEmpty(form.GetCheckedDevice()))
                    {
                        form.UpdateStatus("Device Failure");
                        return;
                    }

                    if (form.Devices.TryGetValue(form.GetCheckedDevice(), out info))
                    {
                        captureManager.FilterByDeviceInfo(info);
                    }

                    liveCamera = true;
                }
            }
            /* Set Module */
            pxcmStatus status = instance.EnableHand(form.GetCheckedModule());
            PXCMHandModule handAnalysis = instance.QueryHand();

            if (status != pxcmStatus.PXCM_STATUS_NO_ERROR || handAnalysis == null)
            {
                form.UpdateStatus("Failed Loading Module");
                return;
            }

PXCMSenseManager.Handler handler = new PXCMSenseManager.Handler();
handler.onModuleProcessedFrame = new PXCMSenseManager.Handler.OnModuleProcessedFrameDelegate(OnNewFrame);

PXCMHandConfiguration handConfiguration = handAnalysis.CreateActiveConfiguration();
PXCMHandData handData = handAnalysis.CreateOutput();

            if (handConfiguration == null)
            {
                form.UpdateStatus("Failed Create Configuration");
                return;
            }
            if (handData==null)
            {
                form.UpdateStatus("Failed Create Output");
                return;
            }

            if (form.getInitGesturesFirstTime() == false)
            {
                int totalNumOfGestures = handConfiguration.QueryGesturesTotalNumber();
                if (totalNumOfGestures > 0)
                {
                    this.form.UpdateGesturesToList("", 0);
                    for (int i = 0; i < totalNumOfGestures; i++)
                    {
                        string gestureName = string.Empty;
                        if (handConfiguration.QueryGestureNameByIndex(i, out gestureName) ==
                            pxcmStatus.PXCM_STATUS_NO_ERROR)
                        {

                                this.form.UpdateGesturesToList(gestureName, i + 1);


                        }

                    }

                    form.setInitGesturesFirstTime(true);
                    form.UpdateGesturesListSize();
                }
            }

            FPSTimer timer = new FPSTimer(form);
            form.UpdateStatus("Init Started");
            if (handAnalysis != null && instance.Init(handler) == pxcmStatus.PXCM_STATUS_NO_ERROR)
            {

PXCMCapture.DeviceInfo dinfo;

                PXCMCapture.Device device = instance.captureManager.device;
                if (device != null)
                {
                    device.QueryDeviceInfo(out dinfo);
                    _maxRange = device.QueryDepthSensorRange().max;

}

                if (handConfiguration != null)
                {
                    handConfiguration.EnableAllAlerts();
                    handConfiguration.EnableSegmentationImage(true);

                    handConfiguration.ApplyChanges();
                    handConfiguration.Update();
                }

                form.UpdateStatus("Streaming");
                int frameCounter = 0;
                int frameNumber = 0;

while (!form.stop)
{

                    string gestureName = form.GetGestureName();
                    if (string.IsNullOrEmpty(gestureName) == false)
                    {
                        if (handConfiguration.IsGestureEnabled(gestureName) == false)
                        {
                            handConfiguration.DisableAllGestures();
                            handConfiguration.EnableGesture(gestureName, true);
                            handConfiguration.ApplyChanges();
                        }
                    }
                    else
                    {
                        handConfiguration.DisableAllGestures();
                        handConfiguration.ApplyChanges();
                    }

                    if (instance.AcquireFrame(true) < pxcmStatus.PXCM_STATUS_NO_ERROR)
                    {
                        break;
                    }

frameCounter++;

if (!DisplayDeviceConnection(!instance.IsConnected()))
{

                        if (handData != null)
                        {
                            handData.Update();
                        }

                        PXCMCapture.Sample sample = instance.QueryHandSample();
                        if (sample != null && sample.depth != null)
                        {
                            DisplayPicture(sample.depth, handData);

                            if (handData != null)
                            {
                                frameNumber = liveCamera ? frameCounter : instance.captureManager.QueryFrameIndex();

                                DisplayJoints(handData);
                                DisplayGesture(handData,frameNumber);
                                DisplayAlerts(handData, frameNumber);
                            }
                            form.UpdatePanel();
                        }
                        timer.Tick();
                    }
                    instance.ReleaseFrame();
                }
            }
            else
            {
                form.UpdateStatus("Init Failed");
                flag = false;
            }
            foreach (PXCMImage pxcmImage in m_images)
            {
                pxcmImage.Dispose();
            }

            // Clean Up
            if (handData != null) handData.Dispose();
            if (handConfiguration != null) handConfiguration.Dispose();
            instance.Close();
            instance.Dispose();

            if (flag)
            {
                form.UpdateStatus("Stopped");
            }

////////try15

pSenseManager = PXCMSenseManager.CreateInstance();

//Then the device instance:

var device = pSenseManager.captureManager.QueryDevice();

//Get the depth photo & data:

            PXCMCapture.Sample photo = pSenseManager.QuerySample();
            PXCMImage depth = photo.depth;
            PXCMImage.ImageData ddata;
            depth.AcquireAccess(PXCMImage.Access.ACCESS_READ, out ddata);

//Set up the projection:

projection = device.CreateProjection();

//Then get the vertices:

PXCMPoint3DF32[] vertices = new PXCMPoint3DF32[depth.info.width * depth.info.height];
projection.QueryVertices(depth, vertices);

/////////////try15

}
}
}

Ok, so in the SimplePipeline

jb455 — Wed, 04 Nov 2015 14:52:44 GMT

Ok, so in the SimplePipeline method you have a SenseManager called instance already (line 413). So anything I've called pSenseManager you can replace with instance.

Then in line 506 you have a device variable which is good.

The rest of my code should go within the while(!form.stop) loop as that's where the video stuff happens: each run of this loop represents a frame of the video feed. So if you stick everything from "PXCMCapture.Sample photo.." to "QueryVertices(depth, vertices);" in that loop it'll update the vertex array each frame for you to do with as you wish.

Thank you Jams You are really

rayan_g_ — Wed, 04 Nov 2015 15:22:35 GMT

Thank you Jams You are really helpful

I did what you said, but I still have errors, also "projection" is undefined

and here is my code and my errors attached

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Drawing;
using System.Drawing.Imaging;
using System.Linq;

namespace hands_viewer.cs
{
    class HandsRecognition
    {
        byte[] LUT;
        private MainForm form;
        private bool _disconnected = false;
        //Queue containing depth image - for synchronization purposes
        private Queue<PXCMImage> m_images;
        private const int NumberOfFramesToDelay = 3;
        private int _framesCounter = 0;
        private float _maxRange;
       


        public HandsRecognition(MainForm form)
        {
            m_images = new Queue<PXCMImage>();
            this.form = form;
            LUT = Enumerable.Repeat((byte)0, 256).ToArray();
            LUT[255] = 1;
        }

        /* Checking if sensor device connect or not */
        private bool DisplayDeviceConnection(bool state)
        {
            if (state)
            {
                if (!_disconnected) form.UpdateStatus("Device Disconnected");
                _disconnected = true;
            }
            else
            {
                if (_disconnected) form.UpdateStatus("Device Reconnected");
                _disconnected = false;
            }
            return _disconnected;
        }

        /* Displaying current frame gestures */
        private void DisplayGesture(PXCMHandData handAnalysis,int frameNumber)
        {

            int firedGesturesNumber = handAnalysis.QueryFiredGesturesNumber();
            string gestureStatusLeft = string.Empty;
            string gestureStatusRight = string.Empty;
            if (firedGesturesNumber == 0)
            {
              
                return;
            }
           
            for (int i = 0; i < firedGesturesNumber; i++)
            {
                PXCMHandData.GestureData gestureData;
                if (handAnalysis.QueryFiredGestureData(i, out gestureData) == pxcmStatus.PXCM_STATUS_NO_ERROR)
                {
                    PXCMHandData.IHand handData;
                    if (handAnalysis.QueryHandDataById(gestureData.handId, out handData) != pxcmStatus.PXCM_STATUS_NO_ERROR)
                        return;
                   
                    PXCMHandData.BodySideType bodySideType = handData.QueryBodySide();
                    if (bodySideType == PXCMHandData.BodySideType.BODY_SIDE_LEFT)
                    {
                        gestureStatusLeft += "Left Hand Gesture: " + gestureData.name;
                    }
                    else if (bodySideType == PXCMHandData.BodySideType.BODY_SIDE_RIGHT)
                    {
                        gestureStatusRight += "Right Hand Gesture: " + gestureData.name;
                    }
                   
                }
                  
            }
            if (gestureStatusLeft == String.Empty)
                form.UpdateInfo("Frame " + frameNumber + ") " + gestureStatusRight + "\n", Color.SeaGreen);
            else
                form.UpdateInfo("Frame " + frameNumber + ") " + gestureStatusLeft + ", " + gestureStatusRight + "\n",Color.SeaGreen);
          
        }

        /* Displaying Depth/Mask Images - for depth image only we use a delay of NumberOfFramesToDelay to sync image with tracking */
        private unsafe void DisplayPicture(PXCMImage depth, PXCMHandData handAnalysis)
        {
            if (depth == null)
                return;

            PXCMImage image = depth;

            //Mask Image
            if (form.GetLabelmapState())
            {
                Bitmap labeledBitmap = null;
                try
                {
                    int numOfHands = handAnalysis.QueryNumberOfHands();

                    PXCMPointI32[][] pointOuter = new PXCMPointI32[numOfHands][];
                    PXCMPointI32[][] pointInner = new PXCMPointI32[numOfHands][];

                    labeledBitmap = new Bitmap(image.info.width, image.info.height, PixelFormat.Format32bppRgb);
                    for (int j = 0; j < numOfHands; j++)
                    {
                        int id;
                        PXCMImage.ImageData data;

                        handAnalysis.QueryHandId(PXCMHandData.AccessOrderType.ACCESS_ORDER_BY_TIME, j, out id);
                        //Get hand by time of appearance
                        PXCMHandData.IHand handData;
                        handAnalysis.QueryHandData(PXCMHandData.AccessOrderType.ACCESS_ORDER_BY_TIME, j, out handData);
                        if (handData != null &&
                            (handData.QuerySegmentationImage(out image) >= pxcmStatus.PXCM_STATUS_NO_ERROR))
                        {
                            if (image.AcquireAccess(PXCMImage.Access.ACCESS_READ, PXCMImage.PixelFormat.PIXEL_FORMAT_Y8,
                                out data) >= pxcmStatus.PXCM_STATUS_NO_ERROR)
                            {
                                Rectangle rect = new Rectangle(0, 0, image.info.width, image.info.height);

                                BitmapData bitmapdata = labeledBitmap.LockBits(rect, ImageLockMode.ReadWrite,
                                    labeledBitmap.PixelFormat);
                                byte* numPtr = (byte*) bitmapdata.Scan0; //dst
                                byte* numPtr2 = (byte*) data.planes[0]; //row
                                int imagesize = image.info.width*image.info.height;
                                byte num2 = (byte) handData.QueryBodySide();

                                byte tmp = 0;
                                for (int i = 0; i < imagesize; i++, numPtr += 4, numPtr2++)
                                {
                                    tmp = (byte) (LUT[numPtr2[0]]*num2*100);
                                    numPtr[0] = (Byte) (tmp | numPtr[0]);
                                    numPtr[1] = (Byte) (tmp | numPtr[1]);
                                    numPtr[2] = (Byte) (tmp | numPtr[2]);
                                    numPtr[3] = 0xff;
                                }

                                labeledBitmap.UnlockBits(bitmapdata);
                                image.ReleaseAccess(data);
                                
                            }

                            if ((form.GetContourState()))
                            {
                                int contourNumber = handData.QueryNumberOfContours();
                                if (contourNumber > 0)
                                {
                                    for (int k = 0; k < contourNumber; ++k)
                                    {
                                        PXCMHandData.IContour contour;
                                        pxcmStatus sts = handData.QueryContour(k, out contour);
                                        if (sts == pxcmStatus.PXCM_STATUS_NO_ERROR)
                                        {
                                            //int contourSize = contour.QuerySize();
                                            if (contour.IsOuter() == true)
                                                contour.QueryPoints(out pointOuter);
                                            else
                                            {
                                                contour.QueryPoints(out pointInner);
                                            }
                                        }
                                    }

                                }
                            }

                        }
                    }
                    if (labeledBitmap != null)
                    {

                        form.DisplayBitmap(labeledBitmap);
                        labeledBitmap.Dispose();
                    }
                    image.Dispose();

                    for (int i = 0; i < numOfHands; i++)
                    {
                        if (form.GetContourState())
                        {
                            if (pointOuter != null && pointOuter.Length > 0)
                                form.DisplayContour(pointOuter, i);
                            if (pointInner != null && pointInner.Length > 0)
                                form.DisplayContour(pointInner, i);
                        }

                    }

                }
                catch (Exception)
                {
                    if (labeledBitmap != null)
                    {
                        labeledBitmap.Dispose();
                    }
                    if (image != null)
                    {
                        image.Dispose();
                    }
                }

            }//end label image

            //Depth Image
            else
            {
                //collecting 3 images inside a queue and displaying the oldest image
                PXCMImage.ImageInfo info;
                PXCMImage image2;

                info = image.QueryInfo();
                image2 = form.g_session.CreateImage(info);
                if (image2 == null) { return; }
                image2.CopyImage(image);
                m_images.Enqueue(image2);
                if (m_images.Count == NumberOfFramesToDelay)
                {
                    Bitmap depthBitmap;
                    try
                    {
                        depthBitmap = new Bitmap(image.info.width, image.info.height, PixelFormat.Format32bppRgb);
                    }
                    catch (Exception)
                    {
                        image.Dispose();
                        PXCMImage queImage = m_images.Dequeue();
                        queImage.Dispose();
                        return;
                    }

                    PXCMImage.ImageData data3;
                    PXCMImage image3 = m_images.Dequeue();
                    if (image3.AcquireAccess(PXCMImage.Access.ACCESS_READ, PXCMImage.PixelFormat.PIXEL_FORMAT_DEPTH, out data3) >= pxcmStatus.PXCM_STATUS_NO_ERROR)
                    {
                        float fMaxValue = _maxRange;
                        byte cVal;

                        Rectangle rect = new Rectangle(0, 0, image.info.width, image.info.height);
                        BitmapData bitmapdata = depthBitmap.LockBits(rect, ImageLockMode.ReadWrite, depthBitmap.PixelFormat);

                        byte* pDst = (byte*)bitmapdata.Scan0;
                        short* pSrc = (short*)data3.planes[0];
                        int size = image.info.width * image.info.height;

                        for (int i = 0; i < size; i++, pSrc++, pDst += 4)
                        {
                            cVal = (byte)((*pSrc) / fMaxValue * 255);
                            if (cVal != 0)
                                cVal = (byte)(255 - cVal);
                           
                            pDst[0] = cVal;
                            pDst[1] = cVal;
                            pDst[2] = cVal;
                            pDst[3] = 255;
                        }
                        try
                        {
                            depthBitmap.UnlockBits(bitmapdata);
                        }
                        catch (Exception)
                        {
                            image3.ReleaseAccess(data3);
                            depthBitmap.Dispose();
                            image3.Dispose();
                            return;
                        }

                        form.DisplayBitmap(depthBitmap);
                        image3.ReleaseAccess(data3);
                    }
                    depthBitmap.Dispose();
                    image3.Dispose();
                }
            }
        }



        /* Displaying current frames hand joints */
        private void DisplayJoints(PXCMHandData handOutput, long timeStamp = 0)
        {
            if (form.GetJointsState() || form.GetSkeletonState())
            {
                //Iterate hands
                PXCMHandData.JointData[][] nodes = new PXCMHandData.JointData[][] { new PXCMHandData.JointData[0x20], new PXCMHandData.JointData[0x20] };
                int numOfHands = handOutput.QueryNumberOfHands();
                for (int i = 0; i < numOfHands; i++)
                {
                    //Get hand by time of appearence
                    //PXCMHandAnalysis.HandData handData = new PXCMHandAnalysis.HandData();
                    PXCMHandData.IHand handData;
                    if (handOutput.QueryHandData(PXCMHandData.AccessOrderType.ACCESS_ORDER_BY_TIME, i, out handData) == pxcmStatus.PXCM_STATUS_NO_ERROR)
                    {
                        if (handData != null)
                        {
                            //Iterate Joints
                            for (int j = 0; j < 0x20; j++)
                            {
                                PXCMHandData.JointData jointData;
                                handData.QueryTrackedJoint((PXCMHandData.JointType)j, out jointData);
                                nodes = jointData;
                                

                               

                            } // end iterating over joints
                        }
                    }
                } // end itrating over hands

                form.DisplayJoints(nodes, numOfHands);
            }
            else
            {
                form.DisplayJoints(null, 0);
            }
        }

        /* Displaying current frame alerts */
        private void DisplayAlerts(PXCMHandData handAnalysis, int frameNumber)
        {
            bool isChanged = false;
            string sAlert = "Alert: ";
            for (int i = 0; i < handAnalysis.QueryFiredAlertsNumber(); i++)
            {
                PXCMHandData.AlertData alertData;
                if (handAnalysis.QueryFiredAlertData(i, out alertData) != pxcmStatus.PXCM_STATUS_NO_ERROR)
                    continue;

                //See PXCMHandAnalysis.AlertData.AlertType for all available alerts
                switch (alertData.label)
                {
                    case PXCMHandData.AlertType.ALERT_HAND_DETECTED:
                        {

                            sAlert += "Hand Detected, ";
                            isChanged = true;
                            break;
                        }
                    case PXCMHandData.AlertType.ALERT_HAND_NOT_DETECTED:
                        {

                            sAlert += "Hand Not Detected, ";
                            isChanged = true;
                            break;
                        }
                    case PXCMHandData.AlertType.ALERT_HAND_CALIBRATED:
                        {

                            sAlert += "Hand Calibrated, ";
                            isChanged = true;
                            break;
                        }
                    case PXCMHandData.AlertType.ALERT_HAND_NOT_CALIBRATED:
                        {

                            sAlert += "Hand Not Calibrated, ";
                            isChanged = true;
                            break;
                        }
                    case PXCMHandData.AlertType.ALERT_HAND_INSIDE_BORDERS:
                        {

                            sAlert += "Hand Inside Border, ";
                            isChanged = true;
                            break;
                        }
                    case PXCMHandData.AlertType.ALERT_HAND_OUT_OF_BORDERS:
                        {

                            sAlert += "Hand Out Of Borders, ";
                            isChanged = true;
                            break;
                        }
                }
            }
            if (isChanged)
            {
                form.UpdateInfo("Frame " + frameNumber + ") " + sAlert + "\n", Color.RoyalBlue);
            }


        }

        public static pxcmStatus OnNewFrame(Int32 mid, PXCMBase module, PXCMCapture.Sample sample)
        {
            return pxcmStatus.PXCM_STATUS_NO_ERROR;
        }

      



        /* Using PXCMSenseManager to handle data */
        public void SimplePipeline()
        {   
            form.UpdateInfo(String.Empty,Color.Black);
            bool liveCamera = false;

            bool flag = true;
            PXCMSenseManager instance = null;
            _disconnected = false;
            instance = form.g_session.CreateSenseManager();
            if (instance == null)
            {
                form.UpdateStatus("Failed creating SenseManager");
                return;
            }

            PXCMCaptureManager captureManager = instance.captureManager;
            if (captureManager != null)
            {
                if (form.GetRecordState())
                {
                    captureManager.SetFileName(form.GetFileName(), true);
                    PXCMCapture.DeviceInfo info;
                    if (form.Devices.TryGetValue(form.GetCheckedDevice(), out info))
                    {
                        captureManager.FilterByDeviceInfo(info);
                    }

                }
                else if (form.GetPlaybackState())
                {
                    captureManager.SetFileName(form.GetFileName(), false);
                }
                else
                {
                    PXCMCapture.DeviceInfo info;
                    if (String.IsNullOrEmpty(form.GetCheckedDevice()))
                    {
                        form.UpdateStatus("Device Failure");
                        return;
                    }

                    if (form.Devices.TryGetValue(form.GetCheckedDevice(), out info))
                    {
                        captureManager.FilterByDeviceInfo(info);
                    }

                    liveCamera = true;
                }
            }
            /* Set Module */
            pxcmStatus status = instance.EnableHand(form.GetCheckedModule());
            PXCMHandModule handAnalysis = instance.QueryHand();

            if (status != pxcmStatus.PXCM_STATUS_NO_ERROR || handAnalysis == null)
            {
                form.UpdateStatus("Failed Loading Module");
                return;
            }

            PXCMSenseManager.Handler handler = new PXCMSenseManager.Handler();
            handler.onModuleProcessedFrame = new PXCMSenseManager.Handler.OnModuleProcessedFrameDelegate(OnNewFrame);


            PXCMHandConfiguration handConfiguration = handAnalysis.CreateActiveConfiguration();
            PXCMHandData handData = handAnalysis.CreateOutput();

            if (handConfiguration == null)
            {
                form.UpdateStatus("Failed Create Configuration");
                return;
            }
            if (handData==null)
            {
                form.UpdateStatus("Failed Create Output");
                return;
            }

            if (form.getInitGesturesFirstTime() == false)
            {
                int totalNumOfGestures = handConfiguration.QueryGesturesTotalNumber();
                if (totalNumOfGestures > 0)
                {
                    this.form.UpdateGesturesToList("", 0);
                    for (int i = 0; i < totalNumOfGestures; i++)
                    {
                        string gestureName = string.Empty;
                        if (handConfiguration.QueryGestureNameByIndex(i, out gestureName) ==
                            pxcmStatus.PXCM_STATUS_NO_ERROR)
                        {
                            
                                this.form.UpdateGesturesToList(gestureName, i + 1);
                            
                            
                        }


                    }
                  
                    form.setInitGesturesFirstTime(true);
                    form.UpdateGesturesListSize();
                }
            }


            FPSTimer timer = new FPSTimer(form);
            form.UpdateStatus("Init Started");
            if (handAnalysis != null && instance.Init(handler) == pxcmStatus.PXCM_STATUS_NO_ERROR)
            {

                PXCMCapture.DeviceInfo dinfo;

                PXCMCapture.Device device = instance.captureManager.device;
                if (device != null)
                {
                    device.QueryDeviceInfo(out dinfo);
                    _maxRange = device.QueryDepthSensorRange().max;

                }
               

                if (handConfiguration != null)
                {
                    handConfiguration.EnableAllAlerts();
                    handConfiguration.EnableSegmentationImage(true);

                    handConfiguration.ApplyChanges();
                    handConfiguration.Update();
                }

                form.UpdateStatus("Streaming");
                int frameCounter = 0;
                int frameNumber = 0;

                while (!form.stop)
                {


                    ////ADD
                    //Get the depth photo & data:

                    PXCMCapture.Sample photo = instance.QuerySample();
                    PXCMImage depth = photo.depth;
                    PXCMImage.ImageData ddata;
                    depth.AcquireAccess(PXCMImage.Access.ACCESS_READ, out ddata);

                    //Set up the projection:

                    projection = device.CreateProjection();

                    //Then get the vertices:

                    PXCMPoint3DF32[] vertices = new PXCMPoint3DF32[depth.info.width * depth.info.height];
                    projection.QueryVertices(depth, vertices);

                    ///ADD


                    string gestureName = form.GetGestureName();
                    if (string.IsNullOrEmpty(gestureName) == false)
                    {
                        if (handConfiguration.IsGestureEnabled(gestureName) == false)
                        {
                            handConfiguration.DisableAllGestures();
                            handConfiguration.EnableGesture(gestureName, true);
                            handConfiguration.ApplyChanges();
                        }
                    }
                    else
                    {
                        handConfiguration.DisableAllGestures();
                        handConfiguration.ApplyChanges();
                    }
                    

                    if (instance.AcquireFrame(true) < pxcmStatus.PXCM_STATUS_NO_ERROR)
                    {
                        break;
                    }

                    frameCounter++;

                    if (!DisplayDeviceConnection(!instance.IsConnected()))
                    {

                        if (handData != null)
                        {
                            handData.Update();
                        }

                        PXCMCapture.Sample sample = instance.QueryHandSample();
                        if (sample != null && sample.depth != null)
                        {
                            DisplayPicture(sample.depth, handData);

                            if (handData != null)
                            {
                                frameNumber = liveCamera ? frameCounter : instance.captureManager.QueryFrameIndex(); 

                                DisplayJoints(handData);
                                DisplayGesture(handData,frameNumber);
                                DisplayAlerts(handData, frameNumber);
                            }
                            form.UpdatePanel();
                        }
                        timer.Tick();
                    }
                    instance.ReleaseFrame();
                }
            }
            else
            {
                form.UpdateStatus("Init Failed");
                flag = false;
            }
            foreach (PXCMImage pxcmImage in m_images)
            {
                pxcmImage.Dispose();
            }

            // Clean Up
            if (handData != null) handData.Dispose();
            if (handConfiguration != null) handConfiguration.Dispose();
            instance.Close();
            instance.Dispose();

            if (flag)
            {
                form.UpdateStatus("Stopped");
            }

            ////////try15


            instance = PXCMSenseManager.CreateInstance();

            //Then the device instance:

            var device = instance.captureManager.QueryDevice();

            




            /////////////try15


        }
    }
}

Delete everything inside //

jb455 — Wed, 04 Nov 2015 15:36:40 GMT

Delete everything inside ///try15, those objects are defined above so you don't need to do anything with them again.

And just put var in front of projection, should work then.

Thank you for your patient

rayan_g_ — Wed, 04 Nov 2015 16:08:45 GMT

Thank you for your patient

It almost works, but still have sum null warning which are stop the program

I have attached the picture and here is my code after change

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Drawing;
using System.Drawing.Imaging;
using System.Linq;

namespace hands_viewer.cs
{
    class HandsRecognition
    {
        byte[] LUT;
        private MainForm form;
        private bool _disconnected = false;
        //Queue containing depth image - for synchronization purposes
        private Queue<PXCMImage> m_images;
        private const int NumberOfFramesToDelay = 3;
        private int _framesCounter = 0;
        private float _maxRange;
       


        public HandsRecognition(MainForm form)
        {
            m_images = new Queue<PXCMImage>();
            this.form = form;
            LUT = Enumerable.Repeat((byte)0, 256).ToArray();
            LUT[255] = 1;
        }

        /* Checking if sensor device connect or not */
        private bool DisplayDeviceConnection(bool state)
        {
            if (state)
            {
                if (!_disconnected) form.UpdateStatus("Device Disconnected");
                _disconnected = true;
            }
            else
            {
                if (_disconnected) form.UpdateStatus("Device Reconnected");
                _disconnected = false;
            }
            return _disconnected;
        }

        /* Displaying current frame gestures */
        private void DisplayGesture(PXCMHandData handAnalysis,int frameNumber)
        {

            int firedGesturesNumber = handAnalysis.QueryFiredGesturesNumber();
            string gestureStatusLeft = string.Empty;
            string gestureStatusRight = string.Empty;
            if (firedGesturesNumber == 0)
            {
              
                return;
            }
           
            for (int i = 0; i < firedGesturesNumber; i++)
            {
                PXCMHandData.GestureData gestureData;
                if (handAnalysis.QueryFiredGestureData(i, out gestureData) == pxcmStatus.PXCM_STATUS_NO_ERROR)
                {
                    PXCMHandData.IHand handData;
                    if (handAnalysis.QueryHandDataById(gestureData.handId, out handData) != pxcmStatus.PXCM_STATUS_NO_ERROR)
                        return;
                   
                    PXCMHandData.BodySideType bodySideType = handData.QueryBodySide();
                    if (bodySideType == PXCMHandData.BodySideType.BODY_SIDE_LEFT)
                    {
                        gestureStatusLeft += "Left Hand Gesture: " + gestureData.name;
                    }
                    else if (bodySideType == PXCMHandData.BodySideType.BODY_SIDE_RIGHT)
                    {
                        gestureStatusRight += "Right Hand Gesture: " + gestureData.name;
                    }
                   
                }
                  
            }
            if (gestureStatusLeft == String.Empty)
                form.UpdateInfo("Frame " + frameNumber + ") " + gestureStatusRight + "\n", Color.SeaGreen);
            else
                form.UpdateInfo("Frame " + frameNumber + ") " + gestureStatusLeft + ", " + gestureStatusRight + "\n",Color.SeaGreen);
          
        }

        /* Displaying Depth/Mask Images - for depth image only we use a delay of NumberOfFramesToDelay to sync image with tracking */
        private unsafe void DisplayPicture(PXCMImage depth, PXCMHandData handAnalysis)
        {
            if (depth == null)
                return;

            PXCMImage image = depth;

            //Mask Image
            if (form.GetLabelmapState())
            {
                Bitmap labeledBitmap = null;
                try
                {
                    int numOfHands = handAnalysis.QueryNumberOfHands();

                    PXCMPointI32[][] pointOuter = new PXCMPointI32[numOfHands][];
                    PXCMPointI32[][] pointInner = new PXCMPointI32[numOfHands][];

                    labeledBitmap = new Bitmap(image.info.width, image.info.height, PixelFormat.Format32bppRgb);
                    for (int j = 0; j < numOfHands; j++)
                    {
                        int id;
                        PXCMImage.ImageData data;

                        handAnalysis.QueryHandId(PXCMHandData.AccessOrderType.ACCESS_ORDER_BY_TIME, j, out id);
                        //Get hand by time of appearance
                        PXCMHandData.IHand handData;
                        handAnalysis.QueryHandData(PXCMHandData.AccessOrderType.ACCESS_ORDER_BY_TIME, j, out handData);
                        if (handData != null &&
                            (handData.QuerySegmentationImage(out image) >= pxcmStatus.PXCM_STATUS_NO_ERROR))
                        {
                            if (image.AcquireAccess(PXCMImage.Access.ACCESS_READ, PXCMImage.PixelFormat.PIXEL_FORMAT_Y8,
                                out data) >= pxcmStatus.PXCM_STATUS_NO_ERROR)
                            {
                                Rectangle rect = new Rectangle(0, 0, image.info.width, image.info.height);

                                BitmapData bitmapdata = labeledBitmap.LockBits(rect, ImageLockMode.ReadWrite,
                                    labeledBitmap.PixelFormat);
                                byte* numPtr = (byte*) bitmapdata.Scan0; //dst
                                byte* numPtr2 = (byte*) data.planes[0]; //row
                                int imagesize = image.info.width*image.info.height;
                                byte num2 = (byte) handData.QueryBodySide();

                                byte tmp = 0;
                                for (int i = 0; i < imagesize; i++, numPtr += 4, numPtr2++)
                                {
                                    tmp = (byte) (LUT[numPtr2[0]]*num2*100);
                                    numPtr[0] = (Byte) (tmp | numPtr[0]);
                                    numPtr[1] = (Byte) (tmp | numPtr[1]);
                                    numPtr[2] = (Byte) (tmp | numPtr[2]);
                                    numPtr[3] = 0xff;
                                }

                                labeledBitmap.UnlockBits(bitmapdata);
                                image.ReleaseAccess(data);
                                
                            }

                            if ((form.GetContourState()))
                            {
                                int contourNumber = handData.QueryNumberOfContours();
                                if (contourNumber > 0)
                                {
                                    for (int k = 0; k < contourNumber; ++k)
                                    {
                                        PXCMHandData.IContour contour;
                                        pxcmStatus sts = handData.QueryContour(k, out contour);
                                        if (sts == pxcmStatus.PXCM_STATUS_NO_ERROR)
                                        {
                                            //int contourSize = contour.QuerySize();
                                            if (contour.IsOuter() == true)
                                                contour.QueryPoints(out pointOuter);
                                            else
                                            {
                                                contour.QueryPoints(out pointInner);
                                            }
                                        }
                                    }

                                }
                            }

                        }
                    }
                    if (labeledBitmap != null)
                    {

                        form.DisplayBitmap(labeledBitmap);
                        labeledBitmap.Dispose();
                    }
                    image.Dispose();

                    for (int i = 0; i < numOfHands; i++)
                    {
                        if (form.GetContourState())
                        {
                            if (pointOuter != null && pointOuter.Length > 0)
                                form.DisplayContour(pointOuter, i);
                            if (pointInner != null && pointInner.Length > 0)
                                form.DisplayContour(pointInner, i);
                        }

                    }

                }
                catch (Exception)
                {
                    if (labeledBitmap != null)
                    {
                        labeledBitmap.Dispose();
                    }
                    if (image != null)
                    {
                        image.Dispose();
                    }
                }

            }//end label image

            //Depth Image
            else
            {
                //collecting 3 images inside a queue and displaying the oldest image
                PXCMImage.ImageInfo info;
                PXCMImage image2;

                info = image.QueryInfo();
                image2 = form.g_session.CreateImage(info);
                if (image2 == null) { return; }
                image2.CopyImage(image);
                m_images.Enqueue(image2);
                if (m_images.Count == NumberOfFramesToDelay)
                {
                    Bitmap depthBitmap;
                    try
                    {
                        depthBitmap = new Bitmap(image.info.width, image.info.height, PixelFormat.Format32bppRgb);
                    }
                    catch (Exception)
                    {
                        image.Dispose();
                        PXCMImage queImage = m_images.Dequeue();
                        queImage.Dispose();
                        return;
                    }

                    PXCMImage.ImageData data3;
                    PXCMImage image3 = m_images.Dequeue();
                    if (image3.AcquireAccess(PXCMImage.Access.ACCESS_READ, PXCMImage.PixelFormat.PIXEL_FORMAT_DEPTH, out data3) >= pxcmStatus.PXCM_STATUS_NO_ERROR)
                    {
                        float fMaxValue = _maxRange;
                        byte cVal;

                        Rectangle rect = new Rectangle(0, 0, image.info.width, image.info.height);
                        BitmapData bitmapdata = depthBitmap.LockBits(rect, ImageLockMode.ReadWrite, depthBitmap.PixelFormat);

                        byte* pDst = (byte*)bitmapdata.Scan0;
                        short* pSrc = (short*)data3.planes[0];
                        int size = image.info.width * image.info.height;

                        for (int i = 0; i < size; i++, pSrc++, pDst += 4)
                        {
                            cVal = (byte)((*pSrc) / fMaxValue * 255);
                            if (cVal != 0)
                                cVal = (byte)(255 - cVal);
                           
                            pDst[0] = cVal;
                            pDst[1] = cVal;
                            pDst[2] = cVal;
                            pDst[3] = 255;
                        }
                        try
                        {
                            depthBitmap.UnlockBits(bitmapdata);
                        }
                        catch (Exception)
                        {
                            image3.ReleaseAccess(data3);
                            depthBitmap.Dispose();
                            image3.Dispose();
                            return;
                        }

                        form.DisplayBitmap(depthBitmap);
                        image3.ReleaseAccess(data3);
                    }
                    depthBitmap.Dispose();
                    image3.Dispose();
                }
            }
        }



        /* Displaying current frames hand joints */
        private void DisplayJoints(PXCMHandData handOutput, long timeStamp = 0)
        {
            if (form.GetJointsState() || form.GetSkeletonState())
            {
                //Iterate hands
                PXCMHandData.JointData[][] nodes = new PXCMHandData.JointData[][] { new PXCMHandData.JointData[0x20], new PXCMHandData.JointData[0x20] };
                int numOfHands = handOutput.QueryNumberOfHands();
                for (int i = 0; i < numOfHands; i++)
                {
                    //Get hand by time of appearence
                    //PXCMHandAnalysis.HandData handData = new PXCMHandAnalysis.HandData();
                    PXCMHandData.IHand handData;
                    if (handOutput.QueryHandData(PXCMHandData.AccessOrderType.ACCESS_ORDER_BY_TIME, i, out handData) == pxcmStatus.PXCM_STATUS_NO_ERROR)
                    {
                        if (handData != null)
                        {
                            //Iterate Joints
                            for (int j = 0; j < 0x20; j++)
                            {
                                PXCMHandData.JointData jointData;
                                handData.QueryTrackedJoint((PXCMHandData.JointType)j, out jointData);
                                nodes = jointData;
                                

                               

                            } // end iterating over joints
                        }
                    }
                } // end itrating over hands

                form.DisplayJoints(nodes, numOfHands);
            }
            else
            {
                form.DisplayJoints(null, 0);
            }
        }

        /* Displaying current frame alerts */
        private void DisplayAlerts(PXCMHandData handAnalysis, int frameNumber)
        {
            bool isChanged = false;
            string sAlert = "Alert: ";
            for (int i = 0; i < handAnalysis.QueryFiredAlertsNumber(); i++)
            {
                PXCMHandData.AlertData alertData;
                if (handAnalysis.QueryFiredAlertData(i, out alertData) != pxcmStatus.PXCM_STATUS_NO_ERROR)
                    continue;

                //See PXCMHandAnalysis.AlertData.AlertType for all available alerts
                switch (alertData.label)
                {
                    case PXCMHandData.AlertType.ALERT_HAND_DETECTED:
                        {

                            sAlert += "Hand Detected, ";
                            isChanged = true;
                            break;
                        }
                    case PXCMHandData.AlertType.ALERT_HAND_NOT_DETECTED:
                        {

                            sAlert += "Hand Not Detected, ";
                            isChanged = true;
                            break;
                        }
                    case PXCMHandData.AlertType.ALERT_HAND_CALIBRATED:
                        {

                            sAlert += "Hand Calibrated, ";
                            isChanged = true;
                            break;
                        }
                    case PXCMHandData.AlertType.ALERT_HAND_NOT_CALIBRATED:
                        {

                            sAlert += "Hand Not Calibrated, ";
                            isChanged = true;
                            break;
                        }
                    case PXCMHandData.AlertType.ALERT_HAND_INSIDE_BORDERS:
                        {

                            sAlert += "Hand Inside Border, ";
                            isChanged = true;
                            break;
                        }
                    case PXCMHandData.AlertType.ALERT_HAND_OUT_OF_BORDERS:
                        {

                            sAlert += "Hand Out Of Borders, ";
                            isChanged = true;
                            break;
                        }
                }
            }
            if (isChanged)
            {
                form.UpdateInfo("Frame " + frameNumber + ") " + sAlert + "\n", Color.RoyalBlue);
            }


        }

        public static pxcmStatus OnNewFrame(Int32 mid, PXCMBase module, PXCMCapture.Sample sample)
        {
            return pxcmStatus.PXCM_STATUS_NO_ERROR;
        }

      



        /* Using PXCMSenseManager to handle data */
        public void SimplePipeline()
        {   
            form.UpdateInfo(String.Empty,Color.Black);
            bool liveCamera = false;

            bool flag = true;
            PXCMSenseManager instance = null;
            _disconnected = false;
            instance = form.g_session.CreateSenseManager();
            if (instance == null)
            {
                form.UpdateStatus("Failed creating SenseManager");
                return;
            }

            PXCMCaptureManager captureManager = instance.captureManager;
            if (captureManager != null)
            {
                if (form.GetRecordState())
                {
                    captureManager.SetFileName(form.GetFileName(), true);
                    PXCMCapture.DeviceInfo info;
                    if (form.Devices.TryGetValue(form.GetCheckedDevice(), out info))
                    {
                        captureManager.FilterByDeviceInfo(info);
                    }

                }
                else if (form.GetPlaybackState())
                {
                    captureManager.SetFileName(form.GetFileName(), false);
                }
                else
                {
                    PXCMCapture.DeviceInfo info;
                    if (String.IsNullOrEmpty(form.GetCheckedDevice()))
                    {
                        form.UpdateStatus("Device Failure");
                        return;
                    }

                    if (form.Devices.TryGetValue(form.GetCheckedDevice(), out info))
                    {
                        captureManager.FilterByDeviceInfo(info);
                    }

                    liveCamera = true;
                }
            }
            /* Set Module */
            pxcmStatus status = instance.EnableHand(form.GetCheckedModule());
            PXCMHandModule handAnalysis = instance.QueryHand();

            if (status != pxcmStatus.PXCM_STATUS_NO_ERROR || handAnalysis == null)
            {
                form.UpdateStatus("Failed Loading Module");
                return;
            }

            PXCMSenseManager.Handler handler = new PXCMSenseManager.Handler();
            handler.onModuleProcessedFrame = new PXCMSenseManager.Handler.OnModuleProcessedFrameDelegate(OnNewFrame);


            PXCMHandConfiguration handConfiguration = handAnalysis.CreateActiveConfiguration();
            PXCMHandData handData = handAnalysis.CreateOutput();

            if (handConfiguration == null)
            {
                form.UpdateStatus("Failed Create Configuration");
                return;
            }
            if (handData==null)
            {
                form.UpdateStatus("Failed Create Output");
                return;
            }

            if (form.getInitGesturesFirstTime() == false)
            {
                int totalNumOfGestures = handConfiguration.QueryGesturesTotalNumber();
                if (totalNumOfGestures > 0)
                {
                    this.form.UpdateGesturesToList("", 0);
                    for (int i = 0; i < totalNumOfGestures; i++)
                    {
                        string gestureName = string.Empty;
                        if (handConfiguration.QueryGestureNameByIndex(i, out gestureName) ==
                            pxcmStatus.PXCM_STATUS_NO_ERROR)
                        {
                            
                                this.form.UpdateGesturesToList(gestureName, i + 1);
                            
                            
                        }


                    }
                  
                    form.setInitGesturesFirstTime(true);
                    form.UpdateGesturesListSize();
                }
            }


            FPSTimer timer = new FPSTimer(form);
            form.UpdateStatus("Init Started");
            if (handAnalysis != null && instance.Init(handler) == pxcmStatus.PXCM_STATUS_NO_ERROR)
            {

                PXCMCapture.DeviceInfo dinfo;

                PXCMCapture.Device device = instance.captureManager.device;
                if (device != null)
                {
                    device.QueryDeviceInfo(out dinfo);
                    _maxRange = device.QueryDepthSensorRange().max;

                }
               

                if (handConfiguration != null)
                {
                    handConfiguration.EnableAllAlerts();
                    handConfiguration.EnableSegmentationImage(true);

                    handConfiguration.ApplyChanges();
                    handConfiguration.Update();
                }

                form.UpdateStatus("Streaming");
                int frameCounter = 0;
                int frameNumber = 0;

                while (!form.stop)
                {


                    ////ADD
                    //Get the depth photo & data:

                    PXCMCapture.Sample photo = instance.QuerySample();
                    PXCMImage depth = photo.depth;
                    PXCMImage.ImageData ddata;
                    depth.AcquireAccess(PXCMImage.Access.ACCESS_READ, out ddata);

                    //Set up the projection:

                    var projection = device.CreateProjection();

                    //Then get the vertices:

                    PXCMPoint3DF32[] vertices = new PXCMPoint3DF32[depth.info.width * depth.info.height];
                    projection.QueryVertices(depth, vertices);

                    ///ADD


                    string gestureName = form.GetGestureName();
                    if (string.IsNullOrEmpty(gestureName) == false)
                    {
                        if (handConfiguration.IsGestureEnabled(gestureName) == false)
                        {
                            handConfiguration.DisableAllGestures();
                            handConfiguration.EnableGesture(gestureName, true);
                            handConfiguration.ApplyChanges();
                        }
                    }
                    else
                    {
                        handConfiguration.DisableAllGestures();
                        handConfiguration.ApplyChanges();
                    }
                    

                    if (instance.AcquireFrame(true) < pxcmStatus.PXCM_STATUS_NO_ERROR)
                    {
                        break;
                    }

                    frameCounter++;

                    if (!DisplayDeviceConnection(!instance.IsConnected()))
                    {

                        if (handData != null)
                        {
                            handData.Update();
                        }

                        PXCMCapture.Sample sample = instance.QueryHandSample();
                        if (sample != null && sample.depth != null)
                        {
                            DisplayPicture(sample.depth, handData);

                            if (handData != null)
                            {
                                frameNumber = liveCamera ? frameCounter : instance.captureManager.QueryFrameIndex(); 

                                DisplayJoints(handData);
                                DisplayGesture(handData,frameNumber);
                                DisplayAlerts(handData, frameNumber);
                            }
                            form.UpdatePanel();
                        }
                        timer.Tick();
                    }
                    instance.ReleaseFrame();
                }
            }
            else
            {
                form.UpdateStatus("Init Failed");
                flag = false;
            }
            foreach (PXCMImage pxcmImage in m_images)
            {
                pxcmImage.Dispose();
            }

            // Clean Up 
            if (handData != null) handData.Dispose();
            if (handConfiguration != null) handConfiguration.Dispose();
            instance.Close();
            instance.Dispose();

            if (flag)
            {
                form.UpdateStatus("Stopped");
            }

            ////////try15

            
            
            /////////////try15


        }
    }
}

Ah, apologies; the //ADD code

jb455 — Thu, 05 Nov 2015 09:34:20 GMT

Ah, apologies; the //ADD code needs to go after the if (instance.AcquireFrame(true) < pxcmStatus.PXCM_STATUS_NO_ERROR) bit. So move it all to either before or after the frameCounter++; line.

I should also mention that you should make sure you dispose or release these objects when you're done with them to avoid memory build ups. So after whatever code you put in to use the vertices (but still within the while loop), stick in:

depth.ReleaseAccess(ddata);
depth.Dispose();
photo.ReleaseImages();
projection.Dispose();

Also, you don't need to post your entire program each time - just the SimplePipeline() section is enough :)

Quote:James N. wrote:

rayan_g_ — Fri, 06 Nov 2015 16:28:00 GMT

James N. wrote:

Ah, apologies; the //ADD code needs to go after the if (instance.AcquireFrame(true) < pxcmStatus.PXCM_STATUS_NO_ERROR) bit. So move it all to either before or after the frameCounter++; line.

I should also mention that you should make sure you dispose or release these objects when you're done with them to avoid memory build ups. So after whatever code you put in to use the vertices (but still within the while loop), stick in:

depth.ReleaseAccess(ddata);
depth.Dispose();
photo.ReleaseImages();
projection.Dispose();

Also, you don't need to post your entire program each time - just the SimplePipeline() section is enough :)

Hi Jams

Thanks again

I did what you said

When I run the code without release the objects by using these four code lines that you wrote, the code is works fine; however, there is no any output for coordinates!!! as shown in attached picure1

When I added these four lines to release the objects, the program stop as shown in the attached picture2.

regards,

////ADD
                    //Get the depth photo & data:

                    PXCMCapture.Sample photo = instance.QuerySample();
                    PXCMImage depth = photo.depth;
                    PXCMImage.ImageData ddata;
                    depth.AcquireAccess(PXCMImage.Access.ACCESS_READ, out ddata);

                    //Set up the projection:

                    var projection = device.CreateProjection();

                    //Then get the vertices:

                    PXCMPoint3DF32[] vertices = new PXCMPoint3DF32[depth.info.width * depth.info.height];
                    projection.QueryVertices(depth, vertices);


                    ///ADD


                    ///ADD2
                    depth.ReleaseAccess(ddata);
                    depth.Dispose();
                    photo.ReleaseImages();
                    projection.Dispose();


                    //ADD2

The vertices array contains

jb455 — Fri, 06 Nov 2015 19:18:50 GMT

The vertices array contains coordinates for each pixel in the depth image.

If you have a point (x,y) from the depth image you're interested in, vertices[depth.info.width * y + x] will return the real-world coordinates for that point.

If you have a colour point, you first have to map that to the depth image as the depth and colour cameras physically point to different places. To do that, use something like:

PXCMPointF32 colourPoint = {x = xCoordinate, y = yCoordinate};
PXCMPointF32[] cp = {colourPoint};
PXCMPointF32[] dp = new PXCMPointF32[cp.Length];
projection.MapColorToDepth(depth, cp, dp);

You can fill cp with as many points as you like, though if you do too many it takes a while to process.
Then vertices[depth.info.width * dp[0].y + dp[0].x] gives you the vertex corresponding to the colour point you wanted.

The above code needs to go somewhere between projection.QueryVertices and projection.dispose in case it's not obvious.

Thanks James

rayan_g_ — Wed, 02 Dec 2015 13:57:00 GMT

Thanks James

so I am done with every think and the code run well without error, but there is no output in the console output.

How can I see the values of the X, Y, Z??

Have you tried using Console

jb455 — Wed, 02 Dec 2015 15:20:31 GMT

Have you tried using Console.WriteLine() or something similar? Or put a breakpoint in at the end of your code to inspect the variables?

Yes I tried

rayan_g_ — Wed, 02 Dec 2015 15:50:00 GMT

Yes I tried

Console.WriteLine(depth.info.width);

it always gives me fixed value (640) which I think it is the resolution, not depth.

also I tried

Console.WriteLine(depth);

and it doesn't work either

In Addition I tried the break point and it is also doesn't work.

Right, so have you done as I

jb455 — Wed, 02 Dec 2015 16:43:16 GMT

Right, so have you done as I suggested in post #14? If you put a breakpoint somewhere after the "MapColourToDepth" line then when it hits put your mouse over dp, you should be able to see the mapped points.

Alternatively you could do something like Console.WriteLine(dp[0].x + ", " + dp[0].y) if you really want it on the console

depth.info.width is the width of the depth image, and depth is an image type so doesn't have a string representation that you can write to the console.

Ah

rayan_g_ — Wed, 02 Dec 2015 17:17:52 GMT

I am talking about my code in post #13 after I fixed some error in the code, it works fine, but I couldn't see any result uin the console or on the video.

I haven't done your post #14 because my code works without these steps; when I tried to do it now, I couldn't define X and Y in my code vertices[depth.info.width * y + x].

C # said it is undefined

Well, it's not working at the

jb455 — Thu, 03 Dec 2015 09:31:01 GMT

Well, it's not working at the moment because you haven't told it which point(s) you want to know about. And nothing's coming up on the console because you haven't told it to put anything there!

If you put a breakpoint after projection.QueryVertices() then mouse over the 'vertices' variable, you should see that it is now populated with lots of points. This is the matrix (or rather, array) of coordinates you asked for in your original post. If you want to know about some points in particular...

As I said in post 14:

If you have (x,y) points in the depth image, to get the real-world coordinates, you use vertices[depth.info.width * y + x].

If you have points in the colour image, you first need to map them to the depth image using the code I gave in #14, then do the step above with the resulting depth points.