- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
/////////////////////////////////////////////////////////// PXCImage::ImageData depthData; if(sample->depth->AcquireAccess(PXCImage::ACCESS_READ, PXCImage::PIXEL_FORMAT_DEPTH, &depthData) >= PXC_STATUS_NO_ERROR) { // depth image height and width: 640 x 480 unsigned short depthValues[640*480]; memcpy(depthValues,depthData.planes[0], sizeof(unsigned short)*640*480); // Necessary to release the access: sample->depth->ReleaseAccess(&depthData); } // use the depthValues array ////////////////////////////////////////////////////////////
And I totally understand that method the way it is. Coming up with that method on my own at this point would be a bit of a challenge. Regardless, I'm glad there are some examples people have shared for storing depth information per pixel from the camera. What I'm interested in however is accessing this information from a Unity script.
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I suppose what you could do is compile the above C++ script into a DLL file and put it into the Plugins.managed folder of your project's Assets panel.
This Unity doc page says how to compile C++ into a DLL and use it in your Unity project.
http://docs.unity3d.com/Manual/UsingDLL.html
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Marty, although I haven't tried it yet, just from my general understanding of the C# scripting in Unity I think that would be an unnecessary step. If the PXCMImage namespace is accessible in C# via the RSSDK there should be a way to easily include these libraries at the beginning of any C# Unity script.
The above code example uses C++ syntax, yes. And I do not know for sure whether C++ libraries can be included in a C# Unity script. Compiling a new DLL in Visual Studio would just make the code more "difficult to work with" in Unity.
I should just be able to say something like the following:
using UnityEngine; using System.Collections; // using [whatever the namespace is for the location of the C# RSSDK library, here?]; public class SomeClassOfMine /* : PXCMSenseManager */ // [?] { void Start() { } void Update() { // algorithm in C# syntax to do what I need here, etc. } }
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
It's my understanding that C++ and C# are very different these days, having drifted apart over time, so converting C++ to C# isn't so easy.
Since you're using Unity, I wonder if using a script like that is over-complicating the problem. If I was approaching it myself, I think I might consider using a 'TrackingAction' or 'TranslateAction' SDK script and track the Z-position value of a particular object, since Z would change as the hand or face moved towards the camera or away from it.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Marty,
That isn't what I want to do at all. You're right though. C++ libraries are not C# libraries. You can totally program in ANSI C in either language however. Like, for example, printf and scanf are just instrinsic parts of the C language core. But that isn't my point. That's totally off topic. What I'm saying is the "algorithm" is perfectly viable. It would be great if I could just say, like...
#include <cstdlib> using namespace std;
...or whatever right at the top of a Unity script, but I don't think that's possible. That's basically what I meant.
PXCMImage exists in the C# library. The RSSDK dynamic link library "libpxcclr.unity.dll" already contains this.
All I have to do is rewrite the syntax. That's what I was thinking, if I didn't say it up there. But I'm very new to this SDK. I would think that you'd have to call, something like...
using UnityEngine; using System.Collections; using System.Collections.Generic; // using RSUnityToolkit; // <---- unnecessary // "libpxcclr.unity" in the plugins.managed folder exposes the RSSDK // I kind of thought for auto complete in Monodevelop to find PXCMImage... // you'd need to declare that you're "using" something. That isn't the case. public class MyClassName : BaseAction // it would inherit from the BaseAction it seems? { void Update() { // algorithm in C# syntax to store pixel depth data per update in an array // and render them out, probably using a Sense AR game object // as re-colored pixels based on the depth } }
Let me make a little more clear what I'm going to do. I'm not tracking an object. I'm re-coloring the pixels based on how far away every pixel's Z-position is from the camera. So...
if (the Z-position is greater than 3 feet away)
colorOfThatPixel = blue;
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I apologize, Lance. C++ code writing is outside of my area of experience. It's definitely something I'm interested in learning when the next RealSense app contest comes around. I'd have no trouble suggesting a way to recolor if it was objects you wanted to recolor rather than pixels.
Don't know if this will be any use, but there's a RealSense documentation page that mentions a C# function called PXCMProjection.QueryUVMap that maps depth to color.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I found this C# script in the RealSense documentation, which maps depth to color coordinates using UV mapping.
// Create the PXCMProjection instance. PXCMProjection projection=device.CreateProjection(); // color and depth image size. PXCMImage.ImageInfo dinfo=depth.info; PXCMImage.ImageInfo cinfo=color.info; // Calculate the UV map. PXCMPointF32[] uvmap=new PXCPointF32[dinfo.width*dinfo.height]; projection.QueryUVMap(depth, uvmap); // Translate depth points PXCPointF32[] uv to color PXCPointF32[] ij for (int i=0;i<uv.Length;i++) { ij.x=uvmap[(int)uv.y*dinfo.width+(int)uv.x].x*cinfo.width; ij.y=uvmap[(int)uv.y*dinfo.width+(int)uv.x].y*cinfo.height; } // Clean up projection.Dispose();
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks Marty, I'll have to read that carefully.
On a side note, I noticed you've linked to the v1.1 version of the documentation. The most recent version of the SDK is the 2014GOLD. Right off the bat when I began searching online for information, I began to realize that there are multiple (and deceptively similar) iterations of the SDK reference guide. The "Perceptual Computing" reference guide is out of date and definitely not the one to use. Between v1.1 and 2014GOLD, I'm not sure what the differences are in their entirety. However, the most correct link at the present time would be here:
No worries. Thanks for your help thus far!

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page