- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi!
I am new with RealSense camera F200.
I am trying export and save the coordinates as a matrix (x,y,z) from any RealSense code for example Hand Console C++ code.
How can I do that?
Regards,
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Rayan,
I recently explained how to do this in another thread here: https://software.intel.com/en-us/forums/realsense/topic/595370 (in C# but it should be mostly the same)
It outputs a single-dimensional array though, not a matrix, so to get a specific (x.y) point you'll need to do vertices[y*depth.info.width + x] (where (x,y) is a depth point - if you have a colour point you'll need to map it like this: https://software.intel.com/en-us/forums/realsense/topic/596309)
James
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
If you have a depth image, you can get X, Y, Z (in mm), with this:
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you for your answer Sir
I tried to use this function, but I couldn't get the (x,y,z) as output and save them as a matrix.
Could you please give me an example; because in the link above there is no any example, just the syntax of this function.
Thank you for your patient.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Why not use the software unity? I think it would be easier to use unity.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Rayan,
I recently explained how to do this in another thread here: https://software.intel.com/en-us/forums/realsense/topic/595370 (in C# but it should be mostly the same)
It outputs a single-dimensional array though, not a matrix, so to get a specific (x.y) point you'll need to do vertices[y*depth.info.width + x] (where (x,y) is a depth point - if you have a colour point you'll need to map it like this: https://software.intel.com/en-us/forums/realsense/topic/596309)
James
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Ariska Hidayat wrote:
Why not use the software unity? I think it would be easier to use unity.
I am haven't use it before, but i will try
Thank you
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
James N. wrote:
Hi Rayan,
I recently explained how to do this in another thread here: https://software.intel.com/en-us/forums/realsense/topic/595370 (in C# but it should be mostly the same)
It outputs a single-dimensional array though, not a matrix, so to get a specific (x.y) point you'll need to do vertices[y*depth.info.width + x] (where (x,y) is a depth point - if you have a colour point you'll need to map it like this: https://software.intel.com/en-us/forums/realsense/topic/596309)
James
Thank you James
I tried to use your code as shown below, but I always get error messages
I don't know if put the code in wrong place ? you can see it between (///try15) comment in the following code.
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Drawing;
using System.Drawing.Imaging;
using System.Linq;
namespace hands_viewer.cs
{
class HandsRecognition
{
byte[] LUT;
private MainForm form;
private bool _disconnected = false;
//Queue containing depth image - for synchronization purposes
private Queue<PXCMImage> m_images;
private const int NumberOfFramesToDelay = 3;
private int _framesCounter = 0;
private float _maxRange;
public HandsRecognition(MainForm form)
{
m_images = new Queue<PXCMImage>();
this.form = form;
LUT = Enumerable.Repeat((byte)0, 256).ToArray();
LUT[255] = 1;
}
/* Checking if sensor device connect or not */
private bool DisplayDeviceConnection(bool state)
{
if (state)
{
if (!_disconnected) form.UpdateStatus("Device Disconnected");
_disconnected = true;
}
else
{
if (_disconnected) form.UpdateStatus("Device Reconnected");
_disconnected = false;
}
return _disconnected;
}
/* Displaying current frame gestures */
private void DisplayGesture(PXCMHandData handAnalysis,int frameNumber)
{
int firedGesturesNumber = handAnalysis.QueryFiredGesturesNumber();
string gestureStatusLeft = string.Empty;
string gestureStatusRight = string.Empty;
if (firedGesturesNumber == 0)
{
return;
}
for (int i = 0; i < firedGesturesNumber; i++)
{
PXCMHandData.GestureData gestureData;
if (handAnalysis.QueryFiredGestureData(i, out gestureData) == pxcmStatus.PXCM_STATUS_NO_ERROR)
{
PXCMHandData.IHand handData;
if (handAnalysis.QueryHandDataById(gestureData.handId, out handData) != pxcmStatus.PXCM_STATUS_NO_ERROR)
return;
PXCMHandData.BodySideType bodySideType = handData.QueryBodySide();
if (bodySideType == PXCMHandData.BodySideType.BODY_SIDE_LEFT)
{
gestureStatusLeft += "Left Hand Gesture: " + gestureData.name;
}
else if (bodySideType == PXCMHandData.BodySideType.BODY_SIDE_RIGHT)
{
gestureStatusRight += "Right Hand Gesture: " + gestureData.name;
}
}
}
if (gestureStatusLeft == String.Empty)
form.UpdateInfo("Frame " + frameNumber + ") " + gestureStatusRight + "\n", Color.SeaGreen);
else
form.UpdateInfo("Frame " + frameNumber + ") " + gestureStatusLeft + ", " + gestureStatusRight + "\n",Color.SeaGreen);
}
/* Displaying Depth/Mask Images - for depth image only we use a delay of NumberOfFramesToDelay to sync image with tracking */
private unsafe void DisplayPicture(PXCMImage depth, PXCMHandData handAnalysis)
{
if (depth == null)
return;
PXCMImage image = depth;
//Mask Image
if (form.GetLabelmapState())
{
Bitmap labeledBitmap = null;
try
{
int numOfHands = handAnalysis.QueryNumberOfHands();
PXCMPointI32[][] pointOuter = new PXCMPointI32[numOfHands][];
PXCMPointI32[][] pointInner = new PXCMPointI32[numOfHands][];
labeledBitmap = new Bitmap(image.info.width, image.info.height, PixelFormat.Format32bppRgb);
for (int j = 0; j < numOfHands; j++)
{
int id;
PXCMImage.ImageData data;
handAnalysis.QueryHandId(PXCMHandData.AccessOrderType.ACCESS_ORDER_BY_TIME, j, out id);
//Get hand by time of appearance
PXCMHandData.IHand handData;
handAnalysis.QueryHandData(PXCMHandData.AccessOrderType.ACCESS_ORDER_BY_TIME, j, out handData);
if (handData != null &&
(handData.QuerySegmentationImage(out image) >= pxcmStatus.PXCM_STATUS_NO_ERROR))
{
if (image.AcquireAccess(PXCMImage.Access.ACCESS_READ, PXCMImage.PixelFormat.PIXEL_FORMAT_Y8,
out data) >= pxcmStatus.PXCM_STATUS_NO_ERROR)
{
Rectangle rect = new Rectangle(0, 0, image.info.width, image.info.height);
BitmapData bitmapdata = labeledBitmap.LockBits(rect, ImageLockMode.ReadWrite,
labeledBitmap.PixelFormat);
byte* numPtr = (byte*) bitmapdata.Scan0; //dst
byte* numPtr2 = (byte*) data.planes[0]; //row
int imagesize = image.info.width*image.info.height;
byte num2 = (byte) handData.QueryBodySide();
byte tmp = 0;
for (int i = 0; i < imagesize; i++, numPtr += 4, numPtr2++)
{
tmp = (byte) (LUT[numPtr2[0]]*num2*100);
numPtr[0] = (Byte) (tmp | numPtr[0]);
numPtr[1] = (Byte) (tmp | numPtr[1]);
numPtr[2] = (Byte) (tmp | numPtr[2]);
numPtr[3] = 0xff;
}
labeledBitmap.UnlockBits(bitmapdata);
image.ReleaseAccess(data);
}
if ((form.GetContourState()))
{
int contourNumber = handData.QueryNumberOfContours();
if (contourNumber > 0)
{
for (int k = 0; k < contourNumber; ++k)
{
PXCMHandData.IContour contour;
pxcmStatus sts = handData.QueryContour(k, out contour);
if (sts == pxcmStatus.PXCM_STATUS_NO_ERROR)
{
//int contourSize = contour.QuerySize();
if (contour.IsOuter() == true)
contour.QueryPoints(out pointOuter
else
{
contour.QueryPoints(out pointInner
}
}
}
}
}
}
}
if (labeledBitmap != null)
{
form.DisplayBitmap(labeledBitmap);
labeledBitmap.Dispose();
}
image.Dispose();
for (int i = 0; i < numOfHands; i++)
{
if (form.GetContourState())
{
if (pointOuter != null && pointOuter.Length > 0)
form.DisplayContour(pointOuter, i);
if (pointInner != null && pointInner.Length > 0)
form.DisplayContour(pointInner, i);
}
}
}
catch (Exception)
{
if (labeledBitmap != null)
{
labeledBitmap.Dispose();
}
if (image != null)
{
image.Dispose();
}
}
}//end label image
//Depth Image
else
{
//collecting 3 images inside a queue and displaying the oldest image
PXCMImage.ImageInfo info;
PXCMImage image2;
info = image.QueryInfo();
image2 = form.g_session.CreateImage(info);
if (image2 == null) { return; }
image2.CopyImage(image);
m_images.Enqueue(image2);
if (m_images.Count == NumberOfFramesToDelay)
{
Bitmap depthBitmap;
try
{
depthBitmap = new Bitmap(image.info.width, image.info.height, PixelFormat.Format32bppRgb);
}
catch (Exception)
{
image.Dispose();
PXCMImage queImage = m_images.Dequeue();
queImage.Dispose();
return;
}
PXCMImage.ImageData data3;
PXCMImage image3 = m_images.Dequeue();
if (image3.AcquireAccess(PXCMImage.Access.ACCESS_READ, PXCMImage.PixelFormat.PIXEL_FORMAT_DEPTH, out data3) >= pxcmStatus.PXCM_STATUS_NO_ERROR)
{
float fMaxValue = _maxRange;
byte cVal;
Rectangle rect = new Rectangle(0, 0, image.info.width, image.info.height);
BitmapData bitmapdata = depthBitmap.LockBits(rect, ImageLockMode.ReadWrite, depthBitmap.PixelFormat);
byte* pDst = (byte*)bitmapdata.Scan0;
short* pSrc = (short*)data3.planes[0];
int size = image.info.width * image.info.height;
for (int i = 0; i < size; i++, pSrc++, pDst += 4)
{
cVal = (byte)((*pSrc) / fMaxValue * 255);
if (cVal != 0)
cVal = (byte)(255 - cVal);
pDst[0] = cVal;
pDst[1] = cVal;
pDst[2] = cVal;
pDst[3] = 255;
}
try
{
depthBitmap.UnlockBits(bitmapdata);
}
catch (Exception)
{
image3.ReleaseAccess(data3);
depthBitmap.Dispose();
image3.Dispose();
return;
}
form.DisplayBitmap(depthBitmap);
image3.ReleaseAccess(data3);
}
depthBitmap.Dispose();
image3.Dispose();
}
}
}
/* Displaying current frames hand joints */
private void DisplayJoints(PXCMHandData handOutput, long timeStamp = 0)
{
if (form.GetJointsState() || form.GetSkeletonState())
{
//Iterate hands
PXCMHandData.JointData[][] nodes = new PXCMHandData.JointData[][] { new PXCMHandData.JointData[0x20], new PXCMHandData.JointData[0x20] };
int numOfHands = handOutput.QueryNumberOfHands();
for (int i = 0; i < numOfHands; i++)
{
//Get hand by time of appearence
//PXCMHandAnalysis.HandData handData = new PXCMHandAnalysis.HandData();
PXCMHandData.IHand handData;
if (handOutput.QueryHandData(PXCMHandData.AccessOrderType.ACCESS_ORDER_BY_TIME, i, out handData) == pxcmStatus.PXCM_STATUS_NO_ERROR)
{
if (handData != null)
{
//Iterate Joints
for (int j = 0; j < 0x20; j++)
{
PXCMHandData.JointData jointData;
handData.QueryTrackedJoint((PXCMHandData.JointType)j, out jointData);
nodes
} // end iterating over joints
}
}
} // end itrating over hands
form.DisplayJoints(nodes, numOfHands);
}
else
{
form.DisplayJoints(null, 0);
}
}
/* Displaying current frame alerts */
private void DisplayAlerts(PXCMHandData handAnalysis, int frameNumber)
{
bool isChanged = false;
string sAlert = "Alert: ";
for (int i = 0; i < handAnalysis.QueryFiredAlertsNumber(); i++)
{
PXCMHandData.AlertData alertData;
if (handAnalysis.QueryFiredAlertData(i, out alertData) != pxcmStatus.PXCM_STATUS_NO_ERROR)
continue;
//See PXCMHandAnalysis.AlertData.AlertType for all available alerts
switch (alertData.label)
{
case PXCMHandData.AlertType.ALERT_HAND_DETECTED:
{
sAlert += "Hand Detected, ";
isChanged = true;
break;
}
case PXCMHandData.AlertType.ALERT_HAND_NOT_DETECTED:
{
sAlert += "Hand Not Detected, ";
isChanged = true;
break;
}
case PXCMHandData.AlertType.ALERT_HAND_CALIBRATED:
{
sAlert += "Hand Calibrated, ";
isChanged = true;
break;
}
case PXCMHandData.AlertType.ALERT_HAND_NOT_CALIBRATED:
{
sAlert += "Hand Not Calibrated, ";
isChanged = true;
break;
}
case PXCMHandData.AlertType.ALERT_HAND_INSIDE_BORDERS:
{
sAlert += "Hand Inside Border, ";
isChanged = true;
break;
}
case PXCMHandData.AlertType.ALERT_HAND_OUT_OF_BORDERS:
{
sAlert += "Hand Out Of Borders, ";
isChanged = true;
break;
}
}
}
if (isChanged)
{
form.UpdateInfo("Frame " + frameNumber + ") " + sAlert + "\n", Color.RoyalBlue);
}
}
public static pxcmStatus OnNewFrame(Int32 mid, PXCMBase module, PXCMCapture.Sample sample)
{
return pxcmStatus.PXCM_STATUS_NO_ERROR;
}
/* Using PXCMSenseManager to handle data */
public void SimplePipeline()
{
form.UpdateInfo(String.Empty,Color.Black);
bool liveCamera = false;
bool flag = true;
PXCMSenseManager instance = null;
_disconnected = false;
instance = form.g_session.CreateSenseManager();
if (instance == null)
{
form.UpdateStatus("Failed creating SenseManager");
return;
}
PXCMCaptureManager captureManager = instance.captureManager;
if (captureManager != null)
{
if (form.GetRecordState())
{
captureManager.SetFileName(form.GetFileName(), true);
PXCMCapture.DeviceInfo info;
if (form.Devices.TryGetValue(form.GetCheckedDevice(), out info))
{
captureManager.FilterByDeviceInfo(info);
}
}
else if (form.GetPlaybackState())
{
captureManager.SetFileName(form.GetFileName(), false);
}
else
{
PXCMCapture.DeviceInfo info;
if (String.IsNullOrEmpty(form.GetCheckedDevice()))
{
form.UpdateStatus("Device Failure");
return;
}
if (form.Devices.TryGetValue(form.GetCheckedDevice(), out info))
{
captureManager.FilterByDeviceInfo(info);
}
liveCamera = true;
}
}
/* Set Module */
pxcmStatus status = instance.EnableHand(form.GetCheckedModule());
PXCMHandModule handAnalysis = instance.QueryHand();
if (status != pxcmStatus.PXCM_STATUS_NO_ERROR || handAnalysis == null)
{
form.UpdateStatus("Failed Loading Module");
return;
}
PXCMSenseManager.Handler handler = new PXCMSenseManager.Handler();
handler.onModuleProcessedFrame = new PXCMSenseManager.Handler.OnModuleProcessedFrameDelegate(OnNewFrame);
PXCMHandConfiguration handConfiguration = handAnalysis.CreateActiveConfiguration();
PXCMHandData handData = handAnalysis.CreateOutput();
if (handConfiguration == null)
{
form.UpdateStatus("Failed Create Configuration");
return;
}
if (handData==null)
{
form.UpdateStatus("Failed Create Output");
return;
}
if (form.getInitGesturesFirstTime() == false)
{
int totalNumOfGestures = handConfiguration.QueryGesturesTotalNumber();
if (totalNumOfGestures > 0)
{
this.form.UpdateGesturesToList("", 0);
for (int i = 0; i < totalNumOfGestures; i++)
{
string gestureName = string.Empty;
if (handConfiguration.QueryGestureNameByIndex(i, out gestureName) ==
pxcmStatus.PXCM_STATUS_NO_ERROR)
{
this.form.UpdateGesturesToList(gestureName, i + 1);
}
}
form.setInitGesturesFirstTime(true);
form.UpdateGesturesListSize();
}
}
FPSTimer timer = new FPSTimer(form);
form.UpdateStatus("Init Started");
if (handAnalysis != null && instance.Init(handler) == pxcmStatus.PXCM_STATUS_NO_ERROR)
{
PXCMCapture.DeviceInfo dinfo;
PXCMCapture.Device device = instance.captureManager.device;
if (device != null)
{
device.QueryDeviceInfo(out dinfo);
_maxRange = device.QueryDepthSensorRange().max;
}
if (handConfiguration != null)
{
handConfiguration.EnableAllAlerts();
handConfiguration.EnableSegmentationImage(true);
handConfiguration.ApplyChanges();
handConfiguration.Update();
}
form.UpdateStatus("Streaming");
int frameCounter = 0;
int frameNumber = 0;
while (!form.stop)
{
string gestureName = form.GetGestureName();
if (string.IsNullOrEmpty(gestureName) == false)
{
if (handConfiguration.IsGestureEnabled(gestureName) == false)
{
handConfiguration.DisableAllGestures();
handConfiguration.EnableGesture(gestureName, true);
handConfiguration.ApplyChanges();
}
}
else
{
handConfiguration.DisableAllGestures();
handConfiguration.ApplyChanges();
}
if (instance.AcquireFrame(true) < pxcmStatus.PXCM_STATUS_NO_ERROR)
{
break;
}
frameCounter++;
if (!DisplayDeviceConnection(!instance.IsConnected()))
{
if (handData != null)
{
handData.Update();
}
PXCMCapture.Sample sample = instance.QueryHandSample();
if (sample != null && sample.depth != null)
{
DisplayPicture(sample.depth, handData);
if (handData != null)
{
frameNumber = liveCamera ? frameCounter : instance.captureManager.QueryFrameIndex();
DisplayJoints(handData);
DisplayGesture(handData,frameNumber);
DisplayAlerts(handData, frameNumber);
}
form.UpdatePanel();
}
timer.Tick();
}
instance.ReleaseFrame();
}
}
else
{
form.UpdateStatus("Init Failed");
flag = false;
}
foreach (PXCMImage pxcmImage in m_images)
{
pxcmImage.Dispose();
}
// Clean Up
if (handData != null) handData.Dispose();
if (handConfiguration != null) handConfiguration.Dispose();
instance.Close();
instance.Dispose();
if (flag)
{
form.UpdateStatus("Stopped");
}
////////try15
pSenseManager = PXCMSenseManager.CreateInstance();
//Then the device instance:
var device = pSenseManager.captureManager.QueryDevice();
//Get the depth photo & data:
PXCMCapture.Sample photo = pSenseManager.QuerySample();
PXCMImage depth = photo.depth;
PXCMImage.ImageData ddata;
depth.AcquireAccess(PXCMImage.Access.ACCESS_READ, out ddata);
//Set up the projection:
projection = device.CreateProjection();
//Then get the vertices:
PXCMPoint3DF32[] vertices = new PXCMPoint3DF32[depth.info.width * depth.info.height];
projection.QueryVertices(depth, vertices);
/////////////try15
}
}
}
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Ok, so in the SimplePipeline method you have a SenseManager called instance already (line 413). So anything I've called pSenseManager you can replace with instance.
Then in line 506 you have a device variable which is good.
The rest of my code should go within the while(!form.stop) loop as that's where the video stuff happens: each run of this loop represents a frame of the video feed. So if you stick everything from "PXCMCapture.Sample photo.." to "QueryVertices(depth, vertices);" in that loop it'll update the vertex array each frame for you to do with as you wish.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you Jams You are really helpful
I did what you said, but I still have errors, also "projection" is undefined
and here is my code and my errors attached
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Drawing;
using System.Drawing.Imaging;
using System.Linq;
namespace hands_viewer.cs
{
class HandsRecognition
{
byte[] LUT;
private MainForm form;
private bool _disconnected = false;
//Queue containing depth image - for synchronization purposes
private Queue<PXCMImage> m_images;
private const int NumberOfFramesToDelay = 3;
private int _framesCounter = 0;
private float _maxRange;
public HandsRecognition(MainForm form)
{
m_images = new Queue<PXCMImage>();
this.form = form;
LUT = Enumerable.Repeat((byte)0, 256).ToArray();
LUT[255] = 1;
}
/* Checking if sensor device connect or not */
private bool DisplayDeviceConnection(bool state)
{
if (state)
{
if (!_disconnected) form.UpdateStatus("Device Disconnected");
_disconnected = true;
}
else
{
if (_disconnected) form.UpdateStatus("Device Reconnected");
_disconnected = false;
}
return _disconnected;
}
/* Displaying current frame gestures */
private void DisplayGesture(PXCMHandData handAnalysis,int frameNumber)
{
int firedGesturesNumber = handAnalysis.QueryFiredGesturesNumber();
string gestureStatusLeft = string.Empty;
string gestureStatusRight = string.Empty;
if (firedGesturesNumber == 0)
{
return;
}
for (int i = 0; i < firedGesturesNumber; i++)
{
PXCMHandData.GestureData gestureData;
if (handAnalysis.QueryFiredGestureData(i, out gestureData) == pxcmStatus.PXCM_STATUS_NO_ERROR)
{
PXCMHandData.IHand handData;
if (handAnalysis.QueryHandDataById(gestureData.handId, out handData) != pxcmStatus.PXCM_STATUS_NO_ERROR)
return;
PXCMHandData.BodySideType bodySideType = handData.QueryBodySide();
if (bodySideType == PXCMHandData.BodySideType.BODY_SIDE_LEFT)
{
gestureStatusLeft += "Left Hand Gesture: " + gestureData.name;
}
else if (bodySideType == PXCMHandData.BodySideType.BODY_SIDE_RIGHT)
{
gestureStatusRight += "Right Hand Gesture: " + gestureData.name;
}
}
}
if (gestureStatusLeft == String.Empty)
form.UpdateInfo("Frame " + frameNumber + ") " + gestureStatusRight + "\n", Color.SeaGreen);
else
form.UpdateInfo("Frame " + frameNumber + ") " + gestureStatusLeft + ", " + gestureStatusRight + "\n",Color.SeaGreen);
}
/* Displaying Depth/Mask Images - for depth image only we use a delay of NumberOfFramesToDelay to sync image with tracking */
private unsafe void DisplayPicture(PXCMImage depth, PXCMHandData handAnalysis)
{
if (depth == null)
return;
PXCMImage image = depth;
//Mask Image
if (form.GetLabelmapState())
{
Bitmap labeledBitmap = null;
try
{
int numOfHands = handAnalysis.QueryNumberOfHands();
PXCMPointI32[][] pointOuter = new PXCMPointI32[numOfHands][];
PXCMPointI32[][] pointInner = new PXCMPointI32[numOfHands][];
labeledBitmap = new Bitmap(image.info.width, image.info.height, PixelFormat.Format32bppRgb);
for (int j = 0; j < numOfHands; j++)
{
int id;
PXCMImage.ImageData data;
handAnalysis.QueryHandId(PXCMHandData.AccessOrderType.ACCESS_ORDER_BY_TIME, j, out id);
//Get hand by time of appearance
PXCMHandData.IHand handData;
handAnalysis.QueryHandData(PXCMHandData.AccessOrderType.ACCESS_ORDER_BY_TIME, j, out handData);
if (handData != null &&
(handData.QuerySegmentationImage(out image) >= pxcmStatus.PXCM_STATUS_NO_ERROR))
{
if (image.AcquireAccess(PXCMImage.Access.ACCESS_READ, PXCMImage.PixelFormat.PIXEL_FORMAT_Y8,
out data) >= pxcmStatus.PXCM_STATUS_NO_ERROR)
{
Rectangle rect = new Rectangle(0, 0, image.info.width, image.info.height);
BitmapData bitmapdata = labeledBitmap.LockBits(rect, ImageLockMode.ReadWrite,
labeledBitmap.PixelFormat);
byte* numPtr = (byte*) bitmapdata.Scan0; //dst
byte* numPtr2 = (byte*) data.planes[0]; //row
int imagesize = image.info.width*image.info.height;
byte num2 = (byte) handData.QueryBodySide();
byte tmp = 0;
for (int i = 0; i < imagesize; i++, numPtr += 4, numPtr2++)
{
tmp = (byte) (LUT[numPtr2[0]]*num2*100);
numPtr[0] = (Byte) (tmp | numPtr[0]);
numPtr[1] = (Byte) (tmp | numPtr[1]);
numPtr[2] = (Byte) (tmp | numPtr[2]);
numPtr[3] = 0xff;
}
labeledBitmap.UnlockBits(bitmapdata);
image.ReleaseAccess(data);
}
if ((form.GetContourState()))
{
int contourNumber = handData.QueryNumberOfContours();
if (contourNumber > 0)
{
for (int k = 0; k < contourNumber; ++k)
{
PXCMHandData.IContour contour;
pxcmStatus sts = handData.QueryContour(k, out contour);
if (sts == pxcmStatus.PXCM_STATUS_NO_ERROR)
{
//int contourSize = contour.QuerySize();
if (contour.IsOuter() == true)
contour.QueryPoints(out pointOuter);
else
{
contour.QueryPoints(out pointInner);
}
}
}
}
}
}
}
if (labeledBitmap != null)
{
form.DisplayBitmap(labeledBitmap);
labeledBitmap.Dispose();
}
image.Dispose();
for (int i = 0; i < numOfHands; i++)
{
if (form.GetContourState())
{
if (pointOuter != null && pointOuter.Length > 0)
form.DisplayContour(pointOuter, i);
if (pointInner != null && pointInner.Length > 0)
form.DisplayContour(pointInner, i);
}
}
}
catch (Exception)
{
if (labeledBitmap != null)
{
labeledBitmap.Dispose();
}
if (image != null)
{
image.Dispose();
}
}
}//end label image
//Depth Image
else
{
//collecting 3 images inside a queue and displaying the oldest image
PXCMImage.ImageInfo info;
PXCMImage image2;
info = image.QueryInfo();
image2 = form.g_session.CreateImage(info);
if (image2 == null) { return; }
image2.CopyImage(image);
m_images.Enqueue(image2);
if (m_images.Count == NumberOfFramesToDelay)
{
Bitmap depthBitmap;
try
{
depthBitmap = new Bitmap(image.info.width, image.info.height, PixelFormat.Format32bppRgb);
}
catch (Exception)
{
image.Dispose();
PXCMImage queImage = m_images.Dequeue();
queImage.Dispose();
return;
}
PXCMImage.ImageData data3;
PXCMImage image3 = m_images.Dequeue();
if (image3.AcquireAccess(PXCMImage.Access.ACCESS_READ, PXCMImage.PixelFormat.PIXEL_FORMAT_DEPTH, out data3) >= pxcmStatus.PXCM_STATUS_NO_ERROR)
{
float fMaxValue = _maxRange;
byte cVal;
Rectangle rect = new Rectangle(0, 0, image.info.width, image.info.height);
BitmapData bitmapdata = depthBitmap.LockBits(rect, ImageLockMode.ReadWrite, depthBitmap.PixelFormat);
byte* pDst = (byte*)bitmapdata.Scan0;
short* pSrc = (short*)data3.planes[0];
int size = image.info.width * image.info.height;
for (int i = 0; i < size; i++, pSrc++, pDst += 4)
{
cVal = (byte)((*pSrc) / fMaxValue * 255);
if (cVal != 0)
cVal = (byte)(255 - cVal);
pDst[0] = cVal;
pDst[1] = cVal;
pDst[2] = cVal;
pDst[3] = 255;
}
try
{
depthBitmap.UnlockBits(bitmapdata);
}
catch (Exception)
{
image3.ReleaseAccess(data3);
depthBitmap.Dispose();
image3.Dispose();
return;
}
form.DisplayBitmap(depthBitmap);
image3.ReleaseAccess(data3);
}
depthBitmap.Dispose();
image3.Dispose();
}
}
}
/* Displaying current frames hand joints */
private void DisplayJoints(PXCMHandData handOutput, long timeStamp = 0)
{
if (form.GetJointsState() || form.GetSkeletonState())
{
//Iterate hands
PXCMHandData.JointData[][] nodes = new PXCMHandData.JointData[][] { new PXCMHandData.JointData[0x20], new PXCMHandData.JointData[0x20] };
int numOfHands = handOutput.QueryNumberOfHands();
for (int i = 0; i < numOfHands; i++)
{
//Get hand by time of appearence
//PXCMHandAnalysis.HandData handData = new PXCMHandAnalysis.HandData();
PXCMHandData.IHand handData;
if (handOutput.QueryHandData(PXCMHandData.AccessOrderType.ACCESS_ORDER_BY_TIME, i, out handData) == pxcmStatus.PXCM_STATUS_NO_ERROR)
{
if (handData != null)
{
//Iterate Joints
for (int j = 0; j < 0x20; j++)
{
PXCMHandData.JointData jointData;
handData.QueryTrackedJoint((PXCMHandData.JointType)j, out jointData);
nodes = jointData;
} // end iterating over joints
}
}
} // end itrating over hands
form.DisplayJoints(nodes, numOfHands);
}
else
{
form.DisplayJoints(null, 0);
}
}
/* Displaying current frame alerts */
private void DisplayAlerts(PXCMHandData handAnalysis, int frameNumber)
{
bool isChanged = false;
string sAlert = "Alert: ";
for (int i = 0; i < handAnalysis.QueryFiredAlertsNumber(); i++)
{
PXCMHandData.AlertData alertData;
if (handAnalysis.QueryFiredAlertData(i, out alertData) != pxcmStatus.PXCM_STATUS_NO_ERROR)
continue;
//See PXCMHandAnalysis.AlertData.AlertType for all available alerts
switch (alertData.label)
{
case PXCMHandData.AlertType.ALERT_HAND_DETECTED:
{
sAlert += "Hand Detected, ";
isChanged = true;
break;
}
case PXCMHandData.AlertType.ALERT_HAND_NOT_DETECTED:
{
sAlert += "Hand Not Detected, ";
isChanged = true;
break;
}
case PXCMHandData.AlertType.ALERT_HAND_CALIBRATED:
{
sAlert += "Hand Calibrated, ";
isChanged = true;
break;
}
case PXCMHandData.AlertType.ALERT_HAND_NOT_CALIBRATED:
{
sAlert += "Hand Not Calibrated, ";
isChanged = true;
break;
}
case PXCMHandData.AlertType.ALERT_HAND_INSIDE_BORDERS:
{
sAlert += "Hand Inside Border, ";
isChanged = true;
break;
}
case PXCMHandData.AlertType.ALERT_HAND_OUT_OF_BORDERS:
{
sAlert += "Hand Out Of Borders, ";
isChanged = true;
break;
}
}
}
if (isChanged)
{
form.UpdateInfo("Frame " + frameNumber + ") " + sAlert + "\n", Color.RoyalBlue);
}
}
public static pxcmStatus OnNewFrame(Int32 mid, PXCMBase module, PXCMCapture.Sample sample)
{
return pxcmStatus.PXCM_STATUS_NO_ERROR;
}
/* Using PXCMSenseManager to handle data */
public void SimplePipeline()
{
form.UpdateInfo(String.Empty,Color.Black);
bool liveCamera = false;
bool flag = true;
PXCMSenseManager instance = null;
_disconnected = false;
instance = form.g_session.CreateSenseManager();
if (instance == null)
{
form.UpdateStatus("Failed creating SenseManager");
return;
}
PXCMCaptureManager captureManager = instance.captureManager;
if (captureManager != null)
{
if (form.GetRecordState())
{
captureManager.SetFileName(form.GetFileName(), true);
PXCMCapture.DeviceInfo info;
if (form.Devices.TryGetValue(form.GetCheckedDevice(), out info))
{
captureManager.FilterByDeviceInfo(info);
}
}
else if (form.GetPlaybackState())
{
captureManager.SetFileName(form.GetFileName(), false);
}
else
{
PXCMCapture.DeviceInfo info;
if (String.IsNullOrEmpty(form.GetCheckedDevice()))
{
form.UpdateStatus("Device Failure");
return;
}
if (form.Devices.TryGetValue(form.GetCheckedDevice(), out info))
{
captureManager.FilterByDeviceInfo(info);
}
liveCamera = true;
}
}
/* Set Module */
pxcmStatus status = instance.EnableHand(form.GetCheckedModule());
PXCMHandModule handAnalysis = instance.QueryHand();
if (status != pxcmStatus.PXCM_STATUS_NO_ERROR || handAnalysis == null)
{
form.UpdateStatus("Failed Loading Module");
return;
}
PXCMSenseManager.Handler handler = new PXCMSenseManager.Handler();
handler.onModuleProcessedFrame = new PXCMSenseManager.Handler.OnModuleProcessedFrameDelegate(OnNewFrame);
PXCMHandConfiguration handConfiguration = handAnalysis.CreateActiveConfiguration();
PXCMHandData handData = handAnalysis.CreateOutput();
if (handConfiguration == null)
{
form.UpdateStatus("Failed Create Configuration");
return;
}
if (handData==null)
{
form.UpdateStatus("Failed Create Output");
return;
}
if (form.getInitGesturesFirstTime() == false)
{
int totalNumOfGestures = handConfiguration.QueryGesturesTotalNumber();
if (totalNumOfGestures > 0)
{
this.form.UpdateGesturesToList("", 0);
for (int i = 0; i < totalNumOfGestures; i++)
{
string gestureName = string.Empty;
if (handConfiguration.QueryGestureNameByIndex(i, out gestureName) ==
pxcmStatus.PXCM_STATUS_NO_ERROR)
{
this.form.UpdateGesturesToList(gestureName, i + 1);
}
}
form.setInitGesturesFirstTime(true);
form.UpdateGesturesListSize();
}
}
FPSTimer timer = new FPSTimer(form);
form.UpdateStatus("Init Started");
if (handAnalysis != null && instance.Init(handler) == pxcmStatus.PXCM_STATUS_NO_ERROR)
{
PXCMCapture.DeviceInfo dinfo;
PXCMCapture.Device device = instance.captureManager.device;
if (device != null)
{
device.QueryDeviceInfo(out dinfo);
_maxRange = device.QueryDepthSensorRange().max;
}
if (handConfiguration != null)
{
handConfiguration.EnableAllAlerts();
handConfiguration.EnableSegmentationImage(true);
handConfiguration.ApplyChanges();
handConfiguration.Update();
}
form.UpdateStatus("Streaming");
int frameCounter = 0;
int frameNumber = 0;
while (!form.stop)
{
////ADD
//Get the depth photo & data:
PXCMCapture.Sample photo = instance.QuerySample();
PXCMImage depth = photo.depth;
PXCMImage.ImageData ddata;
depth.AcquireAccess(PXCMImage.Access.ACCESS_READ, out ddata);
//Set up the projection:
projection = device.CreateProjection();
//Then get the vertices:
PXCMPoint3DF32[] vertices = new PXCMPoint3DF32[depth.info.width * depth.info.height];
projection.QueryVertices(depth, vertices);
///ADD
string gestureName = form.GetGestureName();
if (string.IsNullOrEmpty(gestureName) == false)
{
if (handConfiguration.IsGestureEnabled(gestureName) == false)
{
handConfiguration.DisableAllGestures();
handConfiguration.EnableGesture(gestureName, true);
handConfiguration.ApplyChanges();
}
}
else
{
handConfiguration.DisableAllGestures();
handConfiguration.ApplyChanges();
}
if (instance.AcquireFrame(true) < pxcmStatus.PXCM_STATUS_NO_ERROR)
{
break;
}
frameCounter++;
if (!DisplayDeviceConnection(!instance.IsConnected()))
{
if (handData != null)
{
handData.Update();
}
PXCMCapture.Sample sample = instance.QueryHandSample();
if (sample != null && sample.depth != null)
{
DisplayPicture(sample.depth, handData);
if (handData != null)
{
frameNumber = liveCamera ? frameCounter : instance.captureManager.QueryFrameIndex();
DisplayJoints(handData);
DisplayGesture(handData,frameNumber);
DisplayAlerts(handData, frameNumber);
}
form.UpdatePanel();
}
timer.Tick();
}
instance.ReleaseFrame();
}
}
else
{
form.UpdateStatus("Init Failed");
flag = false;
}
foreach (PXCMImage pxcmImage in m_images)
{
pxcmImage.Dispose();
}
// Clean Up
if (handData != null) handData.Dispose();
if (handConfiguration != null) handConfiguration.Dispose();
instance.Close();
instance.Dispose();
if (flag)
{
form.UpdateStatus("Stopped");
}
////////try15
instance = PXCMSenseManager.CreateInstance();
//Then the device instance:
var device = instance.captureManager.QueryDevice();
/////////////try15
}
}
}
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Delete everything inside ///try15, those objects are defined above so you don't need to do anything with them again.
And just put var in front of projection, should work then.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you for your patient
It almost works, but still have sum null warning which are stop the program
I have attached the picture and here is my code after change
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Drawing;
using System.Drawing.Imaging;
using System.Linq;
namespace hands_viewer.cs
{
class HandsRecognition
{
byte[] LUT;
private MainForm form;
private bool _disconnected = false;
//Queue containing depth image - for synchronization purposes
private Queue<PXCMImage> m_images;
private const int NumberOfFramesToDelay = 3;
private int _framesCounter = 0;
private float _maxRange;
public HandsRecognition(MainForm form)
{
m_images = new Queue<PXCMImage>();
this.form = form;
LUT = Enumerable.Repeat((byte)0, 256).ToArray();
LUT[255] = 1;
}
/* Checking if sensor device connect or not */
private bool DisplayDeviceConnection(bool state)
{
if (state)
{
if (!_disconnected) form.UpdateStatus("Device Disconnected");
_disconnected = true;
}
else
{
if (_disconnected) form.UpdateStatus("Device Reconnected");
_disconnected = false;
}
return _disconnected;
}
/* Displaying current frame gestures */
private void DisplayGesture(PXCMHandData handAnalysis,int frameNumber)
{
int firedGesturesNumber = handAnalysis.QueryFiredGesturesNumber();
string gestureStatusLeft = string.Empty;
string gestureStatusRight = string.Empty;
if (firedGesturesNumber == 0)
{
return;
}
for (int i = 0; i < firedGesturesNumber; i++)
{
PXCMHandData.GestureData gestureData;
if (handAnalysis.QueryFiredGestureData(i, out gestureData) == pxcmStatus.PXCM_STATUS_NO_ERROR)
{
PXCMHandData.IHand handData;
if (handAnalysis.QueryHandDataById(gestureData.handId, out handData) != pxcmStatus.PXCM_STATUS_NO_ERROR)
return;
PXCMHandData.BodySideType bodySideType = handData.QueryBodySide();
if (bodySideType == PXCMHandData.BodySideType.BODY_SIDE_LEFT)
{
gestureStatusLeft += "Left Hand Gesture: " + gestureData.name;
}
else if (bodySideType == PXCMHandData.BodySideType.BODY_SIDE_RIGHT)
{
gestureStatusRight += "Right Hand Gesture: " + gestureData.name;
}
}
}
if (gestureStatusLeft == String.Empty)
form.UpdateInfo("Frame " + frameNumber + ") " + gestureStatusRight + "\n", Color.SeaGreen);
else
form.UpdateInfo("Frame " + frameNumber + ") " + gestureStatusLeft + ", " + gestureStatusRight + "\n",Color.SeaGreen);
}
/* Displaying Depth/Mask Images - for depth image only we use a delay of NumberOfFramesToDelay to sync image with tracking */
private unsafe void DisplayPicture(PXCMImage depth, PXCMHandData handAnalysis)
{
if (depth == null)
return;
PXCMImage image = depth;
//Mask Image
if (form.GetLabelmapState())
{
Bitmap labeledBitmap = null;
try
{
int numOfHands = handAnalysis.QueryNumberOfHands();
PXCMPointI32[][] pointOuter = new PXCMPointI32[numOfHands][];
PXCMPointI32[][] pointInner = new PXCMPointI32[numOfHands][];
labeledBitmap = new Bitmap(image.info.width, image.info.height, PixelFormat.Format32bppRgb);
for (int j = 0; j < numOfHands; j++)
{
int id;
PXCMImage.ImageData data;
handAnalysis.QueryHandId(PXCMHandData.AccessOrderType.ACCESS_ORDER_BY_TIME, j, out id);
//Get hand by time of appearance
PXCMHandData.IHand handData;
handAnalysis.QueryHandData(PXCMHandData.AccessOrderType.ACCESS_ORDER_BY_TIME, j, out handData);
if (handData != null &&
(handData.QuerySegmentationImage(out image) >= pxcmStatus.PXCM_STATUS_NO_ERROR))
{
if (image.AcquireAccess(PXCMImage.Access.ACCESS_READ, PXCMImage.PixelFormat.PIXEL_FORMAT_Y8,
out data) >= pxcmStatus.PXCM_STATUS_NO_ERROR)
{
Rectangle rect = new Rectangle(0, 0, image.info.width, image.info.height);
BitmapData bitmapdata = labeledBitmap.LockBits(rect, ImageLockMode.ReadWrite,
labeledBitmap.PixelFormat);
byte* numPtr = (byte*) bitmapdata.Scan0; //dst
byte* numPtr2 = (byte*) data.planes[0]; //row
int imagesize = image.info.width*image.info.height;
byte num2 = (byte) handData.QueryBodySide();
byte tmp = 0;
for (int i = 0; i < imagesize; i++, numPtr += 4, numPtr2++)
{
tmp = (byte) (LUT[numPtr2[0]]*num2*100);
numPtr[0] = (Byte) (tmp | numPtr[0]);
numPtr[1] = (Byte) (tmp | numPtr[1]);
numPtr[2] = (Byte) (tmp | numPtr[2]);
numPtr[3] = 0xff;
}
labeledBitmap.UnlockBits(bitmapdata);
image.ReleaseAccess(data);
}
if ((form.GetContourState()))
{
int contourNumber = handData.QueryNumberOfContours();
if (contourNumber > 0)
{
for (int k = 0; k < contourNumber; ++k)
{
PXCMHandData.IContour contour;
pxcmStatus sts = handData.QueryContour(k, out contour);
if (sts == pxcmStatus.PXCM_STATUS_NO_ERROR)
{
//int contourSize = contour.QuerySize();
if (contour.IsOuter() == true)
contour.QueryPoints(out pointOuter);
else
{
contour.QueryPoints(out pointInner);
}
}
}
}
}
}
}
if (labeledBitmap != null)
{
form.DisplayBitmap(labeledBitmap);
labeledBitmap.Dispose();
}
image.Dispose();
for (int i = 0; i < numOfHands; i++)
{
if (form.GetContourState())
{
if (pointOuter != null && pointOuter.Length > 0)
form.DisplayContour(pointOuter, i);
if (pointInner != null && pointInner.Length > 0)
form.DisplayContour(pointInner, i);
}
}
}
catch (Exception)
{
if (labeledBitmap != null)
{
labeledBitmap.Dispose();
}
if (image != null)
{
image.Dispose();
}
}
}//end label image
//Depth Image
else
{
//collecting 3 images inside a queue and displaying the oldest image
PXCMImage.ImageInfo info;
PXCMImage image2;
info = image.QueryInfo();
image2 = form.g_session.CreateImage(info);
if (image2 == null) { return; }
image2.CopyImage(image);
m_images.Enqueue(image2);
if (m_images.Count == NumberOfFramesToDelay)
{
Bitmap depthBitmap;
try
{
depthBitmap = new Bitmap(image.info.width, image.info.height, PixelFormat.Format32bppRgb);
}
catch (Exception)
{
image.Dispose();
PXCMImage queImage = m_images.Dequeue();
queImage.Dispose();
return;
}
PXCMImage.ImageData data3;
PXCMImage image3 = m_images.Dequeue();
if (image3.AcquireAccess(PXCMImage.Access.ACCESS_READ, PXCMImage.PixelFormat.PIXEL_FORMAT_DEPTH, out data3) >= pxcmStatus.PXCM_STATUS_NO_ERROR)
{
float fMaxValue = _maxRange;
byte cVal;
Rectangle rect = new Rectangle(0, 0, image.info.width, image.info.height);
BitmapData bitmapdata = depthBitmap.LockBits(rect, ImageLockMode.ReadWrite, depthBitmap.PixelFormat);
byte* pDst = (byte*)bitmapdata.Scan0;
short* pSrc = (short*)data3.planes[0];
int size = image.info.width * image.info.height;
for (int i = 0; i < size; i++, pSrc++, pDst += 4)
{
cVal = (byte)((*pSrc) / fMaxValue * 255);
if (cVal != 0)
cVal = (byte)(255 - cVal);
pDst[0] = cVal;
pDst[1] = cVal;
pDst[2] = cVal;
pDst[3] = 255;
}
try
{
depthBitmap.UnlockBits(bitmapdata);
}
catch (Exception)
{
image3.ReleaseAccess(data3);
depthBitmap.Dispose();
image3.Dispose();
return;
}
form.DisplayBitmap(depthBitmap);
image3.ReleaseAccess(data3);
}
depthBitmap.Dispose();
image3.Dispose();
}
}
}
/* Displaying current frames hand joints */
private void DisplayJoints(PXCMHandData handOutput, long timeStamp = 0)
{
if (form.GetJointsState() || form.GetSkeletonState())
{
//Iterate hands
PXCMHandData.JointData[][] nodes = new PXCMHandData.JointData[][] { new PXCMHandData.JointData[0x20], new PXCMHandData.JointData[0x20] };
int numOfHands = handOutput.QueryNumberOfHands();
for (int i = 0; i < numOfHands; i++)
{
//Get hand by time of appearence
//PXCMHandAnalysis.HandData handData = new PXCMHandAnalysis.HandData();
PXCMHandData.IHand handData;
if (handOutput.QueryHandData(PXCMHandData.AccessOrderType.ACCESS_ORDER_BY_TIME, i, out handData) == pxcmStatus.PXCM_STATUS_NO_ERROR)
{
if (handData != null)
{
//Iterate Joints
for (int j = 0; j < 0x20; j++)
{
PXCMHandData.JointData jointData;
handData.QueryTrackedJoint((PXCMHandData.JointType)j, out jointData);
nodes = jointData;
} // end iterating over joints
}
}
} // end itrating over hands
form.DisplayJoints(nodes, numOfHands);
}
else
{
form.DisplayJoints(null, 0);
}
}
/* Displaying current frame alerts */
private void DisplayAlerts(PXCMHandData handAnalysis, int frameNumber)
{
bool isChanged = false;
string sAlert = "Alert: ";
for (int i = 0; i < handAnalysis.QueryFiredAlertsNumber(); i++)
{
PXCMHandData.AlertData alertData;
if (handAnalysis.QueryFiredAlertData(i, out alertData) != pxcmStatus.PXCM_STATUS_NO_ERROR)
continue;
//See PXCMHandAnalysis.AlertData.AlertType for all available alerts
switch (alertData.label)
{
case PXCMHandData.AlertType.ALERT_HAND_DETECTED:
{
sAlert += "Hand Detected, ";
isChanged = true;
break;
}
case PXCMHandData.AlertType.ALERT_HAND_NOT_DETECTED:
{
sAlert += "Hand Not Detected, ";
isChanged = true;
break;
}
case PXCMHandData.AlertType.ALERT_HAND_CALIBRATED:
{
sAlert += "Hand Calibrated, ";
isChanged = true;
break;
}
case PXCMHandData.AlertType.ALERT_HAND_NOT_CALIBRATED:
{
sAlert += "Hand Not Calibrated, ";
isChanged = true;
break;
}
case PXCMHandData.AlertType.ALERT_HAND_INSIDE_BORDERS:
{
sAlert += "Hand Inside Border, ";
isChanged = true;
break;
}
case PXCMHandData.AlertType.ALERT_HAND_OUT_OF_BORDERS:
{
sAlert += "Hand Out Of Borders, ";
isChanged = true;
break;
}
}
}
if (isChanged)
{
form.UpdateInfo("Frame " + frameNumber + ") " + sAlert + "\n", Color.RoyalBlue);
}
}
public static pxcmStatus OnNewFrame(Int32 mid, PXCMBase module, PXCMCapture.Sample sample)
{
return pxcmStatus.PXCM_STATUS_NO_ERROR;
}
/* Using PXCMSenseManager to handle data */
public void SimplePipeline()
{
form.UpdateInfo(String.Empty,Color.Black);
bool liveCamera = false;
bool flag = true;
PXCMSenseManager instance = null;
_disconnected = false;
instance = form.g_session.CreateSenseManager();
if (instance == null)
{
form.UpdateStatus("Failed creating SenseManager");
return;
}
PXCMCaptureManager captureManager = instance.captureManager;
if (captureManager != null)
{
if (form.GetRecordState())
{
captureManager.SetFileName(form.GetFileName(), true);
PXCMCapture.DeviceInfo info;
if (form.Devices.TryGetValue(form.GetCheckedDevice(), out info))
{
captureManager.FilterByDeviceInfo(info);
}
}
else if (form.GetPlaybackState())
{
captureManager.SetFileName(form.GetFileName(), false);
}
else
{
PXCMCapture.DeviceInfo info;
if (String.IsNullOrEmpty(form.GetCheckedDevice()))
{
form.UpdateStatus("Device Failure");
return;
}
if (form.Devices.TryGetValue(form.GetCheckedDevice(), out info))
{
captureManager.FilterByDeviceInfo(info);
}
liveCamera = true;
}
}
/* Set Module */
pxcmStatus status = instance.EnableHand(form.GetCheckedModule());
PXCMHandModule handAnalysis = instance.QueryHand();
if (status != pxcmStatus.PXCM_STATUS_NO_ERROR || handAnalysis == null)
{
form.UpdateStatus("Failed Loading Module");
return;
}
PXCMSenseManager.Handler handler = new PXCMSenseManager.Handler();
handler.onModuleProcessedFrame = new PXCMSenseManager.Handler.OnModuleProcessedFrameDelegate(OnNewFrame);
PXCMHandConfiguration handConfiguration = handAnalysis.CreateActiveConfiguration();
PXCMHandData handData = handAnalysis.CreateOutput();
if (handConfiguration == null)
{
form.UpdateStatus("Failed Create Configuration");
return;
}
if (handData==null)
{
form.UpdateStatus("Failed Create Output");
return;
}
if (form.getInitGesturesFirstTime() == false)
{
int totalNumOfGestures = handConfiguration.QueryGesturesTotalNumber();
if (totalNumOfGestures > 0)
{
this.form.UpdateGesturesToList("", 0);
for (int i = 0; i < totalNumOfGestures; i++)
{
string gestureName = string.Empty;
if (handConfiguration.QueryGestureNameByIndex(i, out gestureName) ==
pxcmStatus.PXCM_STATUS_NO_ERROR)
{
this.form.UpdateGesturesToList(gestureName, i + 1);
}
}
form.setInitGesturesFirstTime(true);
form.UpdateGesturesListSize();
}
}
FPSTimer timer = new FPSTimer(form);
form.UpdateStatus("Init Started");
if (handAnalysis != null && instance.Init(handler) == pxcmStatus.PXCM_STATUS_NO_ERROR)
{
PXCMCapture.DeviceInfo dinfo;
PXCMCapture.Device device = instance.captureManager.device;
if (device != null)
{
device.QueryDeviceInfo(out dinfo);
_maxRange = device.QueryDepthSensorRange().max;
}
if (handConfiguration != null)
{
handConfiguration.EnableAllAlerts();
handConfiguration.EnableSegmentationImage(true);
handConfiguration.ApplyChanges();
handConfiguration.Update();
}
form.UpdateStatus("Streaming");
int frameCounter = 0;
int frameNumber = 0;
while (!form.stop)
{
////ADD
//Get the depth photo & data:
PXCMCapture.Sample photo = instance.QuerySample();
PXCMImage depth = photo.depth;
PXCMImage.ImageData ddata;
depth.AcquireAccess(PXCMImage.Access.ACCESS_READ, out ddata);
//Set up the projection:
var projection = device.CreateProjection();
//Then get the vertices:
PXCMPoint3DF32[] vertices = new PXCMPoint3DF32[depth.info.width * depth.info.height];
projection.QueryVertices(depth, vertices);
///ADD
string gestureName = form.GetGestureName();
if (string.IsNullOrEmpty(gestureName) == false)
{
if (handConfiguration.IsGestureEnabled(gestureName) == false)
{
handConfiguration.DisableAllGestures();
handConfiguration.EnableGesture(gestureName, true);
handConfiguration.ApplyChanges();
}
}
else
{
handConfiguration.DisableAllGestures();
handConfiguration.ApplyChanges();
}
if (instance.AcquireFrame(true) < pxcmStatus.PXCM_STATUS_NO_ERROR)
{
break;
}
frameCounter++;
if (!DisplayDeviceConnection(!instance.IsConnected()))
{
if (handData != null)
{
handData.Update();
}
PXCMCapture.Sample sample = instance.QueryHandSample();
if (sample != null && sample.depth != null)
{
DisplayPicture(sample.depth, handData);
if (handData != null)
{
frameNumber = liveCamera ? frameCounter : instance.captureManager.QueryFrameIndex();
DisplayJoints(handData);
DisplayGesture(handData,frameNumber);
DisplayAlerts(handData, frameNumber);
}
form.UpdatePanel();
}
timer.Tick();
}
instance.ReleaseFrame();
}
}
else
{
form.UpdateStatus("Init Failed");
flag = false;
}
foreach (PXCMImage pxcmImage in m_images)
{
pxcmImage.Dispose();
}
// Clean Up
if (handData != null) handData.Dispose();
if (handConfiguration != null) handConfiguration.Dispose();
instance.Close();
instance.Dispose();
if (flag)
{
form.UpdateStatus("Stopped");
}
////////try15
/////////////try15
}
}
}
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Ah, apologies; the //ADD code needs to go after the if (instance.AcquireFrame(true) < pxcmStatus.PXCM_STATUS_NO_ERROR) bit. So move it all to either before or after the frameCounter++; line.
I should also mention that you should make sure you dispose or release these objects when you're done with them to avoid memory build ups. So after whatever code you put in to use the vertices (but still within the while loop), stick in:
depth.ReleaseAccess(ddata);
depth.Dispose();
photo.ReleaseImages();
projection.Dispose();
Also, you don't need to post your entire program each time - just the SimplePipeline() section is enough :)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
James N. wrote:
Ah, apologies; the //ADD code needs to go after the if (instance.AcquireFrame(true) < pxcmStatus.PXCM_STATUS_NO_ERROR) bit. So move it all to either before or after the frameCounter++; line.
I should also mention that you should make sure you dispose or release these objects when you're done with them to avoid memory build ups. So after whatever code you put in to use the vertices (but still within the while loop), stick in:
depth.ReleaseAccess(ddata);
depth.Dispose();
photo.ReleaseImages();
projection.Dispose();Also, you don't need to post your entire program each time - just the SimplePipeline() section is enough :)
Hi Jams
Thanks again
I did what you said
When I run the code without release the objects by using these four code lines that you wrote, the code is works fine; however, there is no any output for coordinates!!! as shown in attached picure1
When I added these four lines to release the objects, the program stop as shown in the attached picture2.
regards,
////ADD
//Get the depth photo & data:
PXCMCapture.Sample photo = instance.QuerySample();
PXCMImage depth = photo.depth;
PXCMImage.ImageData ddata;
depth.AcquireAccess(PXCMImage.Access.ACCESS_READ, out ddata);
//Set up the projection:
var projection = device.CreateProjection();
//Then get the vertices:
PXCMPoint3DF32[] vertices = new PXCMPoint3DF32[depth.info.width * depth.info.height];
projection.QueryVertices(depth, vertices);
///ADD
///ADD2
depth.ReleaseAccess(ddata);
depth.Dispose();
photo.ReleaseImages();
projection.Dispose();
//ADD2
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The vertices array contains coordinates for each pixel in the depth image.
If you have a point (x,y) from the depth image you're interested in, vertices[depth.info.width * y + x] will return the real-world coordinates for that point.
If you have a colour point, you first have to map that to the depth image as the depth and colour cameras physically point to different places. To do that, use something like:
PXCMPointF32 colourPoint = {x = xCoordinate, y = yCoordinate}; PXCMPointF32[] cp = {colourPoint}; PXCMPointF32[] dp = new PXCMPointF32[cp.Length]; projection.MapColorToDepth(depth, cp, dp);
You can fill cp with as many points as you like, though if you do too many it takes a while to process.
Then vertices[depth.info.width * dp[0].y + dp[0].x] gives you the vertex corresponding to the colour point you wanted.
The above code needs to go somewhere between projection.QueryVertices and projection.dispose in case it's not obvious.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks James
so I am done with every think and the code run well without error, but there is no output in the console output.
How can I see the values of the X, Y, Z??
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Have you tried using Console.WriteLine() or something similar? Or put a breakpoint in at the end of your code to inspect the variables?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yes I tried
Console.WriteLine(depth.info.width);
it always gives me fixed value (640) which I think it is the resolution, not depth.
also I tried
Console.WriteLine(depth);
and it doesn't work either
In Addition I tried the break point and it is also doesn't work.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Right, so have you done as I suggested in post #14? If you put a breakpoint somewhere after the "MapColourToDepth" line then when it hits put your mouse over dp, you should be able to see the mapped points.
Alternatively you could do something like Console.WriteLine(dp[0].x + ", " + dp[0].y) if you really want it on the console
depth.info.width is the width of the depth image, and depth is an image type so doesn't have a string representation that you can write to the console.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Ah
I am talking about my code in post #13 after I fixed some error in the code, it works fine, but I couldn't see any result uin the console or on the video.
I haven't done your post #14 because my code works without these steps; when I tried to do it now, I couldn't define X and Y in my code vertices[depth.info.width * y + x].
C # said it is undefined
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Well, it's not working at the moment because you haven't told it which point(s) you want to know about. And nothing's coming up on the console because you haven't told it to put anything there!
If you put a breakpoint after projection.QueryVertices() then mouse over the 'vertices' variable, you should see that it is now populated with lots of points. This is the matrix (or rather, array) of coordinates you asked for in your original post. If you want to know about some points in particular...
As I said in post 14:
If you have (x,y) points in the depth image, to get the real-world coordinates, you use vertices[depth.info.width * y + x].
If you have points in the colour image, you first need to map them to the depth image using the code I gave in #14, then do the step above with the resulting depth points.
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page