problems when getting the joint positions of fingers

[a_a_]

I want to get the positions of finger joints. So I copied the code from "Hands Tracking" in "Tutorial_VisualStudioProjectSolution" and changed it a little bit to fit my requirement. However, I have several problems.

1. The code cannot estimate positions of joints in the first two frames. Do I need to add some functions?

2. The code cannot find hands if the depth of hand is lager than about 60cm. However, the "Hands Viewer" can track hand even if the depth is lager than 1m.

3. I have the same problem as https://software.intel.com/en-us/forums/topic/544984 . I have disabled the energy saving settings, but it still does not work. The annoying part is that I have to restart the computer if the program is stuck or the camera will not work.

Could anyone can help me? Thank you.

Here is the code from "Hands Tracking" in "Tutorial_VisualStudioProjectSolution" :

/*******************************************************************************

INTEL CORPORATION PROPRIETARY INFORMATION
This software is supplied under the terms of a license agreement or nondisclosure
agreement with Intel Corporation and may not be copied or disclosed except in
accordance with the terms of that agreement
Copyright(c) 2013-2014 Intel Corporation. All Rights Reserved.

*******************************************************************************/
/* 
Description:
This is the hand tracking and gesture recognition procedural sample that shows how to enable tracking the joints , alert tracking and gesture recognition.

*/

#include <windows.h>
#include "pxcsensemanager.h"
#include "pxchandconfiguration.h"
#include "handtracking_render.h" 

int wmain(int argc, WCHAR* argv[]) {

	// error checking Status
	pxcStatus sts;

	// initialize the util render 
	HandRender *renderer = new HandRender(L"PROCEDURAL HAND TRACKING");

	// create the PXCSenseManager
	PXCSenseManager *psm=0;
	psm = PXCSenseManager::CreateInstance();
	if (!psm) {
		wprintf_s(L"Unable to create the PXCSenseManager\n");
		return 1;
	}

	// enable hand analysis in the multimodal pipeline
	sts = psm->EnableHand();
	if (sts < PXC_STATUS_NO_ERROR) {
		wprintf_s(L"Unable to enable Hand Tracking\n");
		return 2;
	}

	// retrieve hand module if ready - called in the setup stage before AcquireFrame
	PXCHandModule* handAnalyzer = psm->QueryHand();
	if (!handAnalyzer) {
		wprintf_s(L"Unable to retrieve hand results\n");
		return 3;
	}

	// initialize the PXCSenseManager
	if(psm->Init() < PXC_STATUS_NO_ERROR) return 4;

	// retrieves an instance of the PXCHandData interface
	PXCHandData* outputData = handAnalyzer->CreateOutput();

	// retrieves an instance of the PXCHandData PXCHandConfiguration
	PXCHandConfiguration* config = handAnalyzer->CreateActiveConfiguration();

	// enable or disable features in hand module
	config->EnableAlert(PXCHandData::AlertType::ALERT_HAND_DETECTED);
	config->EnableAlert(PXCHandData::AlertType::ALERT_HAND_NOT_DETECTED);
	config->EnableAllGestures();

	//make the config changes effective
	config->ApplyChanges();

	// stream data
	PXCImage *depthIm = NULL; //init depth im
	while (psm->AcquireFrame(true)>=PXC_STATUS_NO_ERROR) {

		//update the output data to the latest availible
		outputData->Update();

		// create data structs for storing data
		PXCHandData::JointData nodes[2][PXCHandData::NUMBER_OF_JOINTS]={};

		// iterate through hands
		int numOfHands = outputData->QueryNumberOfHands();

		for(int i = 0 ; i < numOfHands; i++)
		{
			// get hand joints by time of appearence
			PXCHandData::IHand* handData;
			if(outputData->QueryHandData(PXCHandData::ACCESS_ORDER_BY_TIME,i,handData) == PXC_STATUS_NO_ERROR)
			{
				PXCHandData::JointData jointData;
				// iterate through Joints and get joint data 
				for(int j = 0; j < PXCHandData::NUMBER_OF_JOINTS ; j++)
				{
					handData->QueryTrackedJoint((PXCHandData::JointType)j,jointData);
					nodes = jointData;
				}

			}
		}

		//render Joints
		renderer->DrawJoints(nodes);

		int numOfGestures = outputData->QueryFiredGesturesNumber();
		if(numOfGestures>0){
			// iterate through fired gestures
			for(int i = 0; i < numOfGestures; i++){
				// initialize data
				PXCHandData::GestureData gestureData;
				// get fired gesture data
				if(outputData->QueryFiredGestureData(i,gestureData) == PXC_STATUS_NO_ERROR){
					// get hand data related to fired gesture
					PXCHandData::IHand* handData;
					if(outputData->QueryHandDataById(gestureData.handId,handData) == PXC_STATUS_NO_ERROR){
						// save gesture only if you know that its right/left hand
						if(handData->QueryBodySide() != PXCHandData::BodySideType::BODY_SIDE_UNKNOWN)
							renderer->NotifyGestures(handData->QueryBodySide(),gestureData.name);
					}

				}
			}
		}

		// iterate through Alerts 
		PXCHandData::AlertData alertData;
		for(int i = 0 ; i <outputData->QueryFiredAlertsNumber(); i++)
		{
			sts = outputData->QueryFiredAlertData(i,alertData);
			if(sts==PXC_STATUS_NO_ERROR)
			{
				// Display last alert - see AlertData::Label for all available alerts
				renderer->NotifyAlerts(alertData.label);

			}
		}

		// retrieve all available image samples
		PXCCapture::Sample *sample = psm->QueryHandSample();

		// retrieve the image or frame by type
		depthIm = sample->depth;

		// render the frame
		if (!renderer->RenderFrame(depthIm)) break;

		// release or unlock the current frame to go fetch the next frame
		psm->ReleaseFrame();
	}

	// delete the HandRender instance
	delete renderer;

	//delete the configuration
	config->Release();

	// close the last opened stream and release any session and processing module instances
	psm->Release();

	return 0;
}

[AndreCarlucci]

Hello,

I have some working code tracking the joints here: https://github.com/andrecarlucci/SharpSenses/blob/master/SharpSenses.RealSense/RealSenseCamera.cs

I'ts C#, but it's pretty easy to translate to C++.

Cheers

[AndreCarlucci]

Also,

You don't need to restart the computer if something goes wrong, just restart the windows service named RealSenseDCM and you're good to go :)

[a_a_]

Thank you for the help!

I have tried your suggestion (sometimes I even unplugged the USB), it may work sometimes but it does not always work. I still have to restart the computer.

It that the problem of the sdk or the hardware? I also found that the program may be stuck for several hundred milliseconds and the recorded color image will be darker than the ordinary images. Do you know the reason? Thank you.

Andre Carlucci wrote:

Also,

You don't need to restart the computer if something goes wrong, just restart the windows service named RealSenseDCM and you're good to go :)

[AndreCarlucci]

Sorry, I didn't work directly with images.

I had a similar problem with the camera once, but it was definitely a defective device. I got a new camera and everything worked like a charm.

Try it in a different computer with haswell to confirm that, maybe it's your case too.

[a_a_]

Thanks for the help!

Andre Carlucci wrote:

Sorry, I didn't work directly with images.

I had a similar problem with the camera once, but it was definitely a defective device. I got a new camera and everything worked like a charm.

Try it in a different computer with haswell to confirm that, maybe it's your case too.