How to Add Gestures to Touchless Controller ??

[Khalil_S_]

I want to add gesture ( thumb_up, two finger pinch,...) to Touchless Controller.

Any help appreciated.

[samontab]

As far as I know, that is not supported in the API, but you could implement it yourself.

[Khalil_S_]

okay , 

but how to implement ?? 

[samontab]

One option is to use the RealSense SDK to obtain the position of each finger. Then, use that information over time to define a gesture. Train some machine learning algorithm with this data. Then, you should be able to detect this pattern.

An alternative is to bypass the SDK, and use the raw data to do something similar.

[Khalil_S_]

okay , if using raw streaming ,then how to add cursor on hand movement? any way  to get exact location of moving hand ? 

[samontab]

If you want to move the mouse with the hand movements, that's separable in two projects. One project is the ability to move the mouse with your application. This will depends on the OS you use, etc. This should be pretty easy. The other project is to track the position of the hand. This can be a much harder problem. After you have the two working independently, you will need to start doing some designs, test, etc, to see what speed, type of movement, etc is the best for your application.

[Khalil_S_]

thanks samontab,

From Hand's JoinData i assigned mouse location for mouse cursor movement ,

it works , but not as expected.

when hand moves right , cursor moves left.

also movement of mouse is limited to some area of screen.

How to improve from here ?

[samontab]

Sigh...

Get the maximum possible movement of the hand with some kind of configuration/setup screen.

Apply the mapping from the hand position to the desired mouse position.

[Khalil_S_]

I am getting mirror values,and when setting it to mouse cursor it behaves invert way.

i tried changing using SetMirrorMode as enabled , but it makes raw stream looks mirrored image.