Problem about stabilization and delay

[张_小_]

I use handtracking  to control cursor ,and gesture to control mouse button to manipulate GUI, however, the tracking position is not stable ,so I used smoothing  , but it will lead to delay .  What's more, I want hand moving in a small area and can control a big screen, that means high sensitive, and the  stabilization and delay is a big problem for me.  Are there any solution to fix these problem,  don't say the best solution is to use a real mouse,haha.

[Ron_S_Intel]

Hi 张 小,

Regarding smoothing, note that you have 4 flavors of smoothing - Stabilizer, Weighted, Quadratic and Spring, for mouse cursor I would suggest using the quadratic/spring. if delay is an issue, when initializing the smoother you can play with the init values to find what suits you best.

Regarding tracking area, if you are using unity, you can use the unity tool kit which provides a way of controlling the world box/screen. We are always working on improving the product and will try to do our best to add the above feature to the general sdk.

Be careful of real mouse, it can bite :)

Regards,

Ron

[Basaveswar_K_]

Good technical explanation. Apart from it, I really liked the following statement :)

Ron S. (Intel) wrote:

Be careful of real mouse, it can bite :)

Regards,

Ron

[Ron_S_Intel]

张 小, Another option I can suggest is using the TouchlessController module, this module is specifically designed for cursor purposes and might fit your need.

See c# sample $(RSSDK_PATH)\framework\Csharp\touchless_listbox.cs

Here is a c++ snippet that get event on CursorMove with x,y,z coordinates in the range 0-1
 

// Create an instance of the Session and SenseManager
PXCSession * session = NULL;PXCSenseManager *senseMgr = NULL;
pxcStatus sts = PXCSession_Create(&session);
session->CreateSenseManager(&senseMgr);
// Enable the touchless controller module
senseMgr->EnableTouchlessController();
// Get the system running
senseMgr->Init();

// Inherite from UXEventHandler
class MyEventHandler : public PXCTouchlessController::UXEventHandler{
public:
virtual ~MyEventHandler(){}
virtual  void PXCAPI OnFiredUXEvent(const PXCTouchlessController::UXEventData *uxEventData)
{
 switch (uxEventData->type)
        {
        case PXCTouchlessController::UXEventData::UXEvent_CursorMove:
                 printf("x=%f,y=%f",uxEventData->position.x,xEventData->position.y);
        break;
        }
}
};

// Get touchless controller and register for events
senseMgr->QueryTouchlessController();
MyEventHandler eventHandler;
touchlessController->SubscribeEvent(&eventHandler);

// Main processing loop (run “for ever”)
senseMgr-> StreamFrames(false);
// event are now fired

[张_小_]

hi ,Ron:

Thanks for your reply , I'll try different smoothing methods and the touchless module and choose the best one.

By the way,I have another question: the sdk provide us some methods to get a full hand's position: 1.  PXCMHandData.iHand.QueryMassCenterImage ; 2. PXCMHandData.iHand.QueryTrackedJoint  + JointData.positionWorld or positionImage .

I would like to know which one is faster and more stable, or are there any other good ways.

The touchless_list sample is a good ref for me but I don't know how to use the touchless module to make gesture flexibly, for example,I can't use the "two_fingers_pinch_open" to triggle a mouse button

[MartyG]

Hi 张 小,,

I was working with similar issues to yours when working on my application today.  Trying to have a big effect in a small area in front of the camera, and stabilize the objects (which kept tearing themselves apart once they started moving).

In my own project, I found that the disruption was caused by the Virtual World Box around the objects that had the tracking script in ('TrackingAction' in Unity in my case).  The solution I found was to set all of the values in the 'Real World Box Dimensions' and 'Virtual World Box Dimensions' to zero.  Once I did that, my objects moved super-smoothly.

I second Ron from Intel's suggestion of using a Spring or Quadratic with your smoothing - both are pretty similar to each other, I've found, and both produce good results.  I'm using a Spring with a Smoothing factor of '1' with the 'TrackingAction' script.

[Ron_S_Intel]

Hi 张 小,,

Please see my answers below:

By the way,I have another question: the sdk provide us some methods to get a full hand's position: 1.  PXCMHandData.iHand.QueryMassCenterImage ; 2. PXCMHandData.iHand.QueryTrackedJoint  + JointData.positionWorld or positionImage .

I would like to know which one is faster and more stable, or are there any other good ways.

It depends on what you are trying to achieve, faster - both are the same, stable - for cursor purposes the mass center image would be more stable then a finger tip.

if you don't need the full hand tracking (22 joints) I can also suggest using the PXCBlobExtractor which is more robust - see $(RSSDK_PATH)\sample\maskutils

The touchless_list sample is a good ref for me but I don't know how to use the touchless module to make gesture flexibly, for example,I can't use the "two_fingers_pinch_open" to triggle a mouse button

I would suggest using TouchlessControl module for cursor and HandModule to detect gestures.

Have a nice day,

Ron

[张_小_]

Hi, Ron:

Cool ,new solutions, I think I will try blob later .Thank you .  Here comes the other questions:  1. Have you ever met this problem, my solution is to change the Api Compatibility Level in Unity player settings, but it can only temporarily solve it.

2. I am using the unity toolkit, for the Send Message Action, I found different Gesture trigger send the same trigger value  "gesture detected",so I can't tell them apart in the OnTrigger function.

[张_小_]

For the second question , I think I can use more Send Message Action scripts and more trigger function to solve it

[Ron_S_Intel]

Hi 张 小.

1. It is a common unity issue, I have found possible solutions in  http://forum.unity3d.com/threads/file-access-denied.220913/  

2. The Send Message action uses Unity’s “Send Message” capability and calls the function listed in “FunctionName” with the selected trigger as the argument, your solution is good, you can also expend the toolkit and implement your own action script function.

Regards,

Ron

[涛_季_]

小张，你好啊！

看了半天居然也有中国人在这发帖哈哈哈！