Finger Coordinates question + OpenGL question

[Christopher_C_]

I having a bit of trouble finding how to return the coordinates of a certain finger (the index finger specifically) in the documentation. I'm assuming it has something with JointData, but I only need the coordinats of the last "point" of the index finger. More specifically, I need the index finger acting like the mouse.

Another question:

I'm using OpenGL to develop my app. However, I don't know how make a finger tap gesture act as a mouse click. Does anyone have a clue on what to do? I'm not sure if I'd have to bind gestures to a key/mouse operation because while OpenGL has key/mouse action recognition, it does not have specific recognition for finger gestures. Would I do something such as using an if statement to see if the querygesturedata was a tap, and if it was, then make it act like a mouse tap? (I'm not sure how this would look like in code)

All help is appreciated! Thanks in advance

[Alain_Labrie]

Here is some basic steps on what you need to do:

I) Use QueryTrackedJoint to retrieve current index position with label JOINT_INDEX_TIP.

PXCHandData::JointData data;
iHand->QueryTrackedJoint(PXCHandData::JOINT_INDEX_TIP,data);

2) Store the current position

if(data.confidence > 0)
{
    mFingerPos =data.positionImage;
// or    mFingerPos = data.positionWorld;
}

3) To control user input you can use the Touchless Controller class. Or you can control the mouse cursor by translating the current index finger positionImage to your window's coordinate system and use win32 SendInput function to move the mouse and send click events when a you detect a "Tap" gesture.

Good Luck

[Christopher_C_]

Thanks for the help! I really appreciate it. I'll probably have another question in the near future.

[Christopher_C_]

Say I'm testing the method to retrieve the index finger coordinates in the tutorial's HandAnalysis sample and I want to print the finger's coordinates as soon as it's detected, meaning I want to insert that snippet of code into: 

case PXCHandData::ALERT_HAND_DETECTED:
					{
						wprintf_s(L"Last Alert: Hand Detected\n");
						break;
					}

QUESTION: How would I initialize an iHand*?

I see that in the documentation:

QueryTrackedJoint(JointType label, JointData &data);

means that this function takes in a Joint type enumerator and a jointdata pointer (which I'm assuming I have to insert something like nodes[hand][index finger]. But when I try to create a local iHand* variable, it states that i have not initialized the pointer. This means I'm doing something like this:

case PXCHandData::ALERT_HAND_DETECTED:
					{
						wprintf_s(L"Last Alert: Hand Detected\n");
						PXCHandData::IHand* handData;
handData-> QueryTrackedJoint(PXCHandData::JOINT_INDEX_TIP,nodes[1][4]);//the nodes array address isn't correct, using this as and example

						break;
					}

I know my error is somewhat treating the QueryTrackedJoint as a static function as it kind of seems intuitive to me that this function just returns the tracked joint data to the node I want to store it in. But apparently this is wrong and I'm having a hard time figuring out how to initialize the iHand* for this purpose.

Also, given that we constructed nodes[][] in the the tutorial's HandAnalysis, which array index would we find the index finger tip at?

[Alain_Labrie]

In the code below handData is a PXCHandData* that has already been initialized..

PXCHandData::IHand* iHand;
				
// Get hand by time of appearance 
// The value 0 means that we are only interested in the first hand that was detected
// If you want to track multiple hands then you will need to loop this
if(handData->QueryHandData(PXCHandData::AccessOrderType::ACCESS_ORDER_BY_TIME,0,iHand) == PXC_STATUS_NO_ERROR)
{ 
   for(int j = 0; j < PXCHandData::NUMBER_OF_JOINTS ; j++)
	iHand->QueryTrackedJoint((PXCHandData::JointType)j,nodes[0]);
}

[Christopher_C_]

I cannot stress how much I appreciate your help!

I finally found out I had the wrong tutorial files, meaning that a majority of the code had been changed (which I still don't get because I swear I'm downloading them from the correct link, it might have to do with my browser's cache).

Anyways, I decided to scrap tutorial 3 (hand tracking analysis) and decided the implement my code in tutorial 2 (the raw streams tutorial). However, I'm coming to yet another problem.

I decided to implement the same thing to tutorial 2, which is tracking the position of my index finger. 

Attached below is basically what I changed to the file.

However, when I run the program, I'm stuck with the console window only printing out -107374176.000000

This means that I'm not correctly tracking my index finger (side note: I'm only trying to track the first available hand's index finger).

If possible, could you tell me what I'm doing wrong/help me implement what I'm trying to do? I'm still unfamiliar with intel's realsense library even though I've been reading the documentation.

Thanks for your time! 

[Alain_Labrie]

I took a look at the code you sent me, and you have 2 issues.

1) You were not calling outputData->Update() in your loop

2) It's bad practice but you can declare an array of one object node[1][1]. The problem is that you were trying to access it with the wrong indecies. You need to use node[0][0]. I re-wrote your code and replaced node[1][1] with a variable called myIndex. 

	while (psm->AcquireFrame(true) >= PXC_STATUS_NO_ERROR){

		// This function blocks until all streams are ready (depth and color)
		// if false streams will be unaligned
		if (psm->AcquireFrame(true)<PXC_STATUS_NO_ERROR) break; 

		// retrieve all available image samples
		PXCCapture::Sample *sample = psm->QuerySample();

		// retrieve the image or frame by type from the sample
		colorIm = sample->color;
		depthIm = sample->depth;

		// render the frame
		if (!renderColor->RenderFrame(colorIm)) break;
		if (!renderDepth->RenderFrame(depthIm)) break;

		outputData->Update();
		PXCHandData::JointData myIndex = {};
		PXCHandData::IHand* iHand;
		if (outputData->QueryHandData(PXCHandData::AccessOrderType::ACCESS_ORDER_BY_TIME, 0, iHand) == PXC_STATUS_NO_ERROR)
		{
			iHand->QueryTrackedJoint(PXCHandData::JOINT_INDEX_TIP, myIndex);
		}

		wprintf_s(L"%f\n", myIndex.positionImage.x);

		// release or unlock the current frame to fetch the next frame
		psm->ReleaseFrame();
	}

Good Luck

[Christopher_C_]

Thanks so much for your time and help! If I happen to place in the contest drinks will be on me