I can't get device pointer

[SKim61]

I am trying to use projection API however whenever I try to access device with

PXCCapture::Device* device = captureManager->QueryDevice();

PXCProjection* projection = device->CreateProjection();

Then I got null from it. How can I get device pointer?

[MartyG]

Advanced stream programming is not my field of expertise, so I will link stream expert jb455 into this discussion to bring it to his attention. In the meantime, the most compatible source of existing knowledge i could find that related to your question is in the link below. I hope it can provide some useful insight.

https://software.intel.com/en-us/forums/realsense/topic/596423 Cannot get Device Instance

[SKim61]

Thank you I follow the instruction but can't do it.

You know how to add # include "util_cmdline.h" in visual studio?

[MartyG]

I do not develop in Visual Studio sadly, as I am a Unity game engine specialist. I would ask though if you are using Visual Studio C++. This is because C# scripts do not need includes in the header section of the script.

[SKim61]

Do you have any projection example code with C# ? Also, can you pointer your C# blogs or relative good explanation blogs or sources? Using realSense sdk is not easy. How to match c++ sdk definition with C# ? is it almost same? From the example I don't get how to use api's with it.

[jb455]

Yes, it is hard to get started. I've been using it for about a year and a half now; there was much less help about when I was learning so I just had to work it out by myself mostly! There were some very helpful people on the old forum too. You can mostly piece it together by going through https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?doc_devguide_introduction.html the documentation and looking at the C++ sample but it does take time. As I said, if you're stuck with how to use a particular method in the projection api, search the forum for the method name and with any luck it'll be one someone has already asked about and had it explained. If not, feel free to post another question (ideally in a new thread to make it easier for future people to find) and hopefully someone will be able to help you.

[MartyG]

Thanks for the mention of the Unity guides, JB.

Spk921, here's the link to the index of guides if you haven't found it yet.

https://software.intel.com/en-us/forums/realsense/topic/676139 Index of Marty G's RealSense Unity How-To Guides

It's a very useful link to bookmark because you can also use it as a launching pad for searches of the old RealSense forum with the 'Powered by Google' search box, as a lot of valuable information is archived on that former forum.

[SKim61]

Thank you this is really helpful.

However, where can I find sample source code that are listed in Unity Application and your posts?

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?samples_toolkit.html Intel® RealSense™ SDK 2016 R2 Documentation

Your posts says about how to use Unity but I need how to bind with realSense

For example here https://software.intel.com/en-us/forums/realsense/topic/556743 Unity tip: Controlling Avatar Arms And Legs With A Single RealSense Camera

help using Unity but don't know how to get "hand detected" icon. It seems that it need some script to add component.

But there are no explain how to right code with realSense SDK.

Anyway, thank you for leading me. I will take a look in depth.

[MartyG]

The R2 SDK comes with a program called the Unity Toolkit. If you run it whilst you have a project open in Unity then it imports the RealSense camera library files and tools into that project. These tools include a folder of pre-made 'Action' scripts for purposes such as hand / face / object tracking and triggering actions that you can just drop into a Unity object without having to do script programming.

The pre-made script that can be used like the example mentioned to control objects such as an avatars arms and legs, is called TrackingAction.

[jb455]

Does the camera streaming work without the device/projection stuff? If not, it may be a driver/hardware issue.

One thing which may help is looking at the samples to see how they populate the device selection drop down menu, then how they get it to use that device (instead of another one if you have multiple plugged in). That way, you'll definitely know that the device is available before starting to stream with it.

[SKim61]

Thank you. I take a looked but it is over complicated.

PXCCaptureManager *cm = sm->QueryCaptureManager();PXCCapture::Device *cp = cm->QueryDevice();

PXCProjection* projection = cp->CreateProjection();

How can I get device with above code?

Why I can't access to the device?

I can get depth and color frames without device pointer but need to get UV map to match color and depth images.

Because two cameras are depart from it. Thanks.

[jb455]

Before those 3 lines, you use sm->EnableStreams(..) and sm->Init(), right? I think it's at the Init() point that it chooses the device and enables it, so you won't be able to query the device before that. Does it work if you do sm->captureManager->device->CreateProjection(); ?

[SKim61]

# include

# include "RealSense/SenseManager.h"

# include "RealSense/SampleReader.h"

# include "pxcsensemanager.h"

# include "pxcprojection.h"

# include "pxcmetadata.h"

# include "pxcsession.h"

# include "pxccapture.h"

//# include "util_cmdline.h"

//# include "util_render.h"

# include

# include

# include

# include

# include

# include

//Code adapted from the Kinect v2 samples

HRESULT SaveBitmapToFile(BYTE* pBitmapBits, LONG lWidth, LONG lHeight, WORD wBitsPerPixel, LPCWSTR lpszFilePath)

{

DWORD dwByteCount = lWidth * lHeight * (wBitsPerPixel / 8);

BITMAPINFOHEADER bmpInfoHeader = { 0 };

bmpInfoHeader.biSize = sizeof(BITMAPINFOHEADER); // Size of the header

bmpInfoHeader.biBitCount = wBitsPerPixel; // Bit count

bmpInfoHeader.biCompression = BI_RGB; // Standard RGB, no compression

bmpInfoHeader.biWidth = lWidth; // Width in pixels

bmpInfoHeader.biHeight = -lHeight; // Height in pixels, negative indicates it's stored right-side-up

bmpInfoHeader.biPlanes = 1; // Default

bmpInfoHeader.biSizeImage = dwByteCount; // Image size in bytes

BITMAPFILEHEADER bfh = { 0 };

bfh.bfType = 0x4D42; // 'M''B', indicates bitmap

bfh.bfOffBits = bmpInfoHeader.biSize + sizeof(BITMAPFILEHEADER); // Offset to the start of pixel data

bfh.bfSize = bfh.bfOffBits + bmpInfoHeader.biSizeImage; // Size of image + headers

// Create the file on disk to write to

HANDLE hFile = CreateFileW(lpszFilePath, GENERIC_WRITE, 0, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);

// Return if error opening file

if (NULL == hFile)

{

return E_ACCESSDENIED;

}

DWORD dwBytesWritten = 0;

// Write the bitmap file header

if (!WriteFile(hFile, &bfh, sizeof(bfh), &dwBytesWritten, NULL))

{

CloseHandle(hFile);

return E_FAIL;

}

// Write the bitmap info header

if (!WriteFile(hFile, &bmpInfoHeader, sizeof(bmpInfoHeader), &dwBytesWritten, NULL))

{

CloseHandle(hFile);

return E_FAIL;

}

// Write the RGB Data

if (!WriteFile(hFile, pBitmapBits, bmpInfoHeader.biSizeImage, &dwBytesWritten, NULL))

{

CloseHandle(hFile);

return E_FAIL;

}

// Close the file

CloseHandle(hFile);

return S_OK;

}

// https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?doc_essential_strong_synchronization.html

int wmain(int argc, WCHAR* argv[]) {

int NFrames = 400;

if (argc > 1) {

NFrames = _wtoi(argv[1]);

}

std::cout << "Capturing " << NFrames << " frames";

int width = 320, height = 240;

BYTE* colorFrames = new BYTE[NFrames*width*height * 3];

float* depthFrames = new float[NFrames*width*height];

PXCPointF32* UVMaps = new PXCPointF32[NFrames*width*height];

PXCSenseManager* sm = PXCSenseManager::CreateInstance();

PXCSession* session = sm->QuerySession();

// Select the color and depth streams

sm->EnableStream(PXCCapture::STREAM_TYPE_COLOR, width, height, 60.0f);

sm->EnableStream(PXCCapture::STREAM_TYPE_DEPTH, width, height, 60.0f);

// Initialize and Stream Samples

sm->Init();

//Get projection operator

PXCCaptureManager *cm = sm->QueryCaptureManager();

PXCCapture::Device *cp = cm->QueryDevice();

PXCProjection* projection = cp->CreateProjection();

std::ofstream fout;

fout.open("B-timings.txt");

fout << "n, color, depth\n";

//Step 1: Throw away the first 50 frames because the camera seems to be warming up

for (int i = 0; i < 50; i++) {

if (sm->AcquireFrame(true)

sm->ReleaseFrame();

}

//Step 2: Capture frames

for (int i = 0; i < NFrames; i++) {

std::cout << "Capturing frame " << i << "\n";

fout << i << ", ";

// This function blocks until both samples are ready

if (sm->AcquireFrame(true)

//Retrieve the color and depth samples aligned

PXCCapture::Sample *sample = sm->QuerySample();

//Capture the color frame

PXCImage* c = sample->color;

pxcI64 ctimestamp = c->QueryTimeStamp();

fout << ctimestamp << ", ";

PXCImage::ImageData imageData;

// https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/manuals_image_and_audio_data.html

c->AcquireAccess(PXCImage::ACCESS_READ, PXCImage::PIXEL_FORMAT_RGB24, &imageData);

memcpy(&colorFrames[i*width*height * 3], imageData.planes[0], sizeof(BYTE)*width*height * 3);

c->ReleaseAccess(&imageData);

//Capture the depth frame

PXCImage* d = sample->depth;

pxcI64 dtimestamp = d->QueryTimeStamp();

fout << dtimestamp << "\n";

PXCImage::ImageData depthData;

d->AcquireAccess(PXCImage::ACCESS_READ, PXCImage::PIXEL_FORMAT_DEPTH_F32, &depthData);

memcpy(&depthFrames[i*width*height], depthDat...

[SKim61]

This is the code which I am using it. Yes I did after init()

[jb455]

Hm, the only thing I have different to you is I use device = sm.captureManager.QueryDevice(); - does that make any difference?

[SKim61]

I can't do "DOT" since it's a pointer. Can you share your code that should be faster.

[SKim61]

Also what should I set visual studio setting to use

//# include "util_cmdline.h"

//# include "util_render.h"

this headers?

[jb455]

Sorry, I'm using C# so you'll have to change it to C++ syntax. device = sm->captureManager->QueryDevice(); is it?

[SKim61]

Yes C# should be better! For me because I would like to use it for Unity Binding. I was not able to see proper function app definition from C# so I was planning to do CPP and convert to .dll. Can you send it by e-mail? mailto:spkim921@gmail.com spkim921@gmail.com.

Also, it should be helpful if you could point out C# example code. It is really lack of information just looking at realSense example since it has only one.

[jb455]

There are a few C# (WinForms) samples included in the SDK, though unfortunately none of them use projections You can use the RawStreams sample as a base to get basic stuff working then use the docs/modify the C++ projections sample to go from there. My company doesn't allow sharing all my code but I have posted a few snippets showing how to do various things over the years - just search for the method name you're struggling with in the forum and you may find something to help (I did put a lot in the old forum too but that's hard to access now).

If you're doing it in unity C# , MartyG has created a series of guides to help people get started - search in the forum for that too.

[jb455]

As for the headers, no idea. Check the C++ samples!