Hand Recognition on a surface

[Dimitrios_A_]

Hello,

Does Intel RealSense recognize your hand when they are on a surface?

I would like to use the camera facing down to the desk and the desk surface as a touchscreen.

Thank you.

[MartyG · Accepted solution]

The HP Sprout PC, which uses a special non-standard version of RealSense and an overhead camera, sounds ideal for what you are trying to achieve.

However, if you don't want to pay nearly $2000 for a Sprout then you may be able to emulate its functions wit a regular F200 desktop camera.  There are a few problems to overcome though.  Firstly, the camera would be unlikely to be able to see your hands properly if both the camera and hands were on the desk.  The best option may therefore be to hang the F200 camera from the top of your monitor using the flexible stand on the base of it, and have it pointing downwards.

Assuming that your hands are far away enough from the camera for it to be able to track them, the next problem is that as the palms will be flat down on the desk, you may have to set the tracking's start condition to 'Always' (instead of Hands Detected), since the camera cannot see the palms in order for tracking to be activated by the Hands Detected condition.

The next problem is giving the camera a hand joint to track.  You Cannot use the palm (the Center joint) to track, as it is facing down on the desk.  You want to choose a joint that has maximum visibility to the camera so that tracking does not stall if the joint disappears from its view.  If the camera is looking downwards then your best choice may be one of the Base finger joints (the knuckles behind the finger) or the Wrist joints (the big bumps just behind the hand where the start of your wrist is).

If your application is going to involve tracking finger-bending, Joint 1, also known as JT1 - the next joint up from the base - may be a better choice than the Base knuckles.

Because you are tracking from top-down instead of the hands facing front-on to the camera, you may find that the steering behavior is different (e.g an object moving sidewards instead of up-down).  

If top-down tracking does not work out for you, another option may be to have the camera set maybe half a meter back from the desktop and have the camera track the finger-tip joints as the hands are face-down.  The camera and hands would have to be placed in such a way that they are level though, since if the hands dropped down beneath the camera's line of sight then the tracking would stall.

[Dimitrios_A_]

Thank you for the quick response!

I will propably buy the F200 and try to make it work.

[Elad_P_]

Hi Marty and Dimitrios,

I just came across this post when looking for the same thing.

Marty, I didn't understand your suggestion regarding the "giving the camera a hand joint to track". When enabling hand tracking, it tracks the whole hand (and all fingers). How can one give it a specific joint to track.

Dimitrios, did you manage to get it to work?