RealSense R200: How resolve call to QueryJoints (C#) causing exception?

[Joseph_W_1]

Hello:

I am using the RealSense SDK + R200 camera for prototyping in a personal mobility project.

This task involves replicating then subsequently extending the source code provided by Intel Application Engineer Brian Brown in this project on the Intel Developer Zone: 

DIY Pan-Tilt Person Tracking with Intel® RealSense™ Camera

Thus far I have been able to successfully replicate the entire project - all face and body tracking with the R200 works well, as does acquisition of data for servo control (which in our case will be further used to control companion robotics). FWIW, the dev environment is VS 2015 running on an Intel Compute Stick (Product code : BOXSTK2M3W64CC, 64-bit Windows 10 Pro)

The next step was to add code to query skeleton joint information for the purpose of determining distance from the origin (the R200) to a given body joint using the world coordinate for that joint.

I can successfully call QuerySkeletonJoints(), then use the number of joints detected to allocate an array of SkeletonPoint structs. However, when I make the call to QueryJoints(), passing in the array of SkeletonPoint structs as an argument, I get this error:

An unhandled exception of type 'System.AccessViolationException' occurred in libpxcclr.cs.dll

The code is identical to the source code in the Update() method in the the  MainWindow.xaml.cs file of the source code Brian provided on the project page referenced above. The change I made was to insert the following after face tracking data was acquired, but before the call to Render(), as shown below; it is at the call to personJoint.QueryJoints( jointArray ) that the exception occurs:

               if( trackedPerson != null ) {

                    PXCMPersonTrackingData.PersonJoints personJoint = trackedPerson.QuerySkeletonJoints();
                    int njoints = personJoint.QueryNumJoints();
                    // Update joint count in tracked person object
                    myTrackedPerson.JointsDetected = njoints;
                    PXCMPersonTrackingData.PersonJoints.SkeletonPoint[] jointArray = new PXCMPersonTrackingData.PersonJoints.SkeletonPoint[njoints];
                    
                    if (njoints > 0)
                    {
                         if (personJoint.QueryJoints( jointArray ) )
                        {
                            for (int i = 0; i < njoints; i++)
                            {
                                if (jointArray.confidenceImage > 80)
                                {
                                    if (jointArray.jointType == PXCMPersonTrackingData.PersonJoints.JointType.JOINT_HEAD)
                                    {
                                        // ...do what needed done here...
                                    }
                                }
                            }
                        }
                    }

                }

===

Any assistance someone could provide would be much appreciated. Disclaimer(s):  (1) this is my 1st post to IDF, so if this posting was not made in the manner it should have been, please provide useful, constructive feedback and (2) I've been away for C# for a while doing Android/Java/embedded development, so maybe I just missed something obvious in C#.

Thank you!

[mohan_g_1]

Hey Joseph,

You need to add following statement in post build event command line at properties.

 if "$(Platform)" == "x86" ( copy /y "$(RSSDK_DIR)\bin\win32\libpxccpp2c.dll" "$(TargetDir)" ) else ( copy /y "$(RSSDK_DIR)\bin\x64\libpxccpp2c.dll" "$(TargetDir)" )

Make sure you have libpxcccppc2c.dll in above specified directory.

Thanks,

-MohanG

[Joseph_W_1]

Mohan:

Thank you for the info - I had confirmed that post-build event statement was in place.

Regards,

   - joseph

[Joseph_W_1]

For anyone interested, here's how I resolved this problem:

Research led me to this blog posting by Mike Tualty, a Developer/Evangelist for Microsoft that had run into a problem virtually identical to the one described in my original posting herein:

https://mtaulty.com/2016/05/11/windows-10-wpf-realsense-sr300-person-tracking-continued/

After perusing the blog I contacted Mike for further clarification, here are the key points from his response:

>Yes, I made a minor hack to the SDK in order to get that API call to work for me.
>
>If you look in your equivalent folder to;
> 
>C:\Program Files (x86)\Intel\RSSDK\framework\common\pxcclr.cs\
> 
>And note that pxcclr.cs is a folder which is a bit of a weird name then you’ll 
>find Visual Studio project and solution files and a src folder. Open up the 
>solution file, change the code as per my blog post and then rebuild the 
>library do produce the libpxcclr.cs.dll.
> 
>Now, you need to reference that DLL from your WPF project (by using the add 
>reference dialog) rather than the one that shipped with the RealSense SDK.

I made the described code modifications, re-built and re-referenced the dll as instructed, and the problem was resolved.

Hope this helps,

   - joseph

[mohan_g_1]

Joseph:

Glad that your issue is now resolved. By the way did you able to retrieve skeleton joints for r200?

In my case am always getting 0 for query number of joints.

Thanks,

-MohanG

[Joseph_W_1]

Mohan:

I doesn't seem to me that skeleton joints is working.

In the vast majority of frames processed, the number of joints detected is 0.

When joints are detected, the number is always 6.

Fwiw, here is how I set up and enabled skeleton joints: 

// Enable person tracking
senseManager.EnablePersonTracking();
personModule = senseManager.QueryPersonTracking();
PXCMPersonTrackingConfiguration personConfig = personModule.QueryConfiguration();
personConfig.SetTrackedAngles(PXCMPersonTrackingConfiguration.TrackingAngles.TRACKING_ANGLES_ALL);

// Enable skeleton joint tracking
PXCMPersonTrackingConfiguration.SkeletonJointsConfiguration skeletonConfiguration = personConfig.QuerySkeletonJoints();
skeletonConfiguration.SetMaxTrackedPersons(1);
skeletonConfiguration.SetTrackingArea(PXCMPersonTrackingConfiguration.SkeletonJointsConfiguration.SkeletonMode.AREA_FULL_BODY_ROUGH);
skeletonConfiguration.Enable();

[mohan_g_1]

Joseph,

I did configure my skeleton joints set up like yours but still am getting 0 number of joints,
is there anything else that i have missed?

Thanks,

-MohanG

[mohan_g_1]

Joseph,

I did configure my skeleton joints set up like yours but still am getting 0 number of joints,
is there anything else that i have missed?

Thanks,

-MohanG