R200 Processing and Camera Streams

[MPerr9]

Hi, I am creating a sketch in Processing and seem to have some issues around the https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/pxcimage.html PXCMImage.https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/imagedata_pxcimage.html ImageData.ToPImage class.

My code looks as such...

import intel.rssdk.*;

PXCMSenseManager sm = null;

//int nframes = 0;

PImage imgDepth;

void setup() {

size(640, 480);

// Create a SenseManager instance

PXCMSenseManager sm=PXCMSenseManager.CreateInstance();

// Select the color and depth streams

sm.EnableStream(PXCMCapture.StreamType.STREAM_TYPE_COLOR,640,480,30);

sm.EnableStream(PXCMCapture.StreamType.STREAM_TYPE_DEPTH,320,240,30);

// Initialize and Stream Samples

sm.Init();

for (;;) {

// This function blocks until both samples are ready

if (sm.AcquireFrame(true).isError()) break;

// retrieve the samples

PXCMCapture.Sample sample=sm.QuerySample();

// work on the samples: sample.color & sample.depth

// ...

if (sample.depth != null) {

PXCMImage.ImageData dData = new PXCMImage.ImageData();

sample.depth.AcquireAccess(PXCMImage.Access.ACCESS_READ,PXCMImage.PixelFormat.PIXEL_FORMAT_RGB32, dData);

// imgDepth = dData.ToPImage(0, imgDepth);

// PXCMImage.ImageData.ToPImage(0, imgDepth);

sample.depth.ReleaseAccess(dData);

println(dData);

}

//hoping to convert the PXCMImage data to something drawable in canvas. String fail

//String depth=sample.depth;

//loadImage("depth");

}

// go fetching the next samples

sm.ReleaseFrame();

}

void draw() {

//need to get data from sample.depth here.

image("imgDepth", 0, 0);

}

// Close down

void dispose(){

sm.close();

}

I can see the camera is initialized, and that it is filling up data in memory by using the print line, but ToPImage does not seem to work as expected.

Another method is a previous sample was

// imgDepth = dData.ToPImage(0, imgDepth);

but also no luck that, or even getting the sample to work correctly

I am sure it is something quite simple that I am overlooking due to being new to Processing / Java. If anyone has any insights that would truly be appreciated.

Best, and thank you!

Matt

[MPerr9]

After a bit more time, it seems the biggest error seems to be around

imgDepth = dData.ToPImage(0, imgDepth);

This function in the examples just does not seem to work with R2 and Processing 3.3

Can anyone confirm? Thanks,

Matt

[MartyG]

In the release notes for 2016 R2, there is an entry in the Known Issues for Processing in the SR300 section of the notes that says:

Although the reference is in the SR300 section of the notes, it may also be relevant to the R200 camera, and you therefore need to use a 2.x version of Processing instead of the 3.3 version you are currently using.

[JTsan1]

I am using Processing 2.1.2 and it still does not work with R2. The processing IDE states that "The function ToPImage(int, PImage) does not exist.". I decompile the "libpxcclr_processing.jar". And, the ToPImage function indeed does not exist.

[MartyG]

The user Matt who began this discussion also experienced this error and wrote about it on the Processing forum. A solution was not found in that case though.

https://forum.processing.org/two/discussion/21989/intel-realsense-r200-obtain-depth-image-in-canvas Intel Realsense R200 Obtain Depth Image in Canvas - Processing 2.x and 3.x Forum

This is Intel's official Processing documentation page for using ToPImage with RealSense.

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?topimage_imagedata_pxcmimage.html Intel® RealSense™ SDK 2016 R2 Documentation

Regarding not being able to find ToPImage in the code: it would not be the first time that a function in the documentation is not actually currently implemented in the SDK software. Someone else in 2016 who also believed that ToPImage had been removed posted a workaround. They said: "I ended up just copying the function into my regular code and avoid the use of 'toPimage' altogether (And just use the java library jar):" Here's the script they posted.

public PImage ToPImageCopy(int var1, PImage var2, PXCMImage.ImageData cData) {

var2.loadPixels();

cData.ToIntArray(var1, var2.pixels);

var2.updatePixels();

return var2;

}

[JTsan1]

Thanks for the reply. I found that the workaround does work. Thanks a lot.

Link to the source of the workaround:

https://software.intel.com/en-us/forums/realsense/topic/611243 [Processing][Java] toPimage fails with Segmentation output.

[MartyG]

Awesome! I'm very glad it worked for you. Thanks too for the link so that others in the community can benefit from it. Good luck with the rest of your project!