Accurate hand/skeletal tracking options for D4xx series

[WTatt]

Hi All,

We are looking for further input on accurate and reliable hand and skeletal tracking solutions for the D4xx series - would be interested in hearing other peoples opinions.Westa

[MartyG · Accepted solution]

Skeletal tracking for the 400 Series is available through software called Nuitrack.

https://www.youtube.com/watch?v=gMPtV4NXtUo Intel RealSense D415/D435 and Nuitrack skeletal tracking SDK replace Kinect SDK - YouTube

[AZhu6]

Hello, is there a way can do something like the link below by using D415

https://www.youtube.com/watch%3Fv%3DRQ-kAoaNc60 Accurate, Robust, and Flexible Real-time Hand Tracking - YouTube

I want to focus on hand tracking to control the bionic arm. Any advice is welcome

[WTatt]

Hey Aaron,

There is nothing specifically in the public domain right now for the d4xx series regarding accurate hand tracking of fingers - BUT there are a number of OpenCV workflows that can give reasonable approximations.

Have a look at openCV examples on youtube for example - https://dev.to/amarlearning/finger-detection-and-tracking-using-opencv-and-python-586m Finger Detection and Tracking using OpenCV and Python - DEV Community

and the opencv docs have some notes on tracking algorithms https://docs.opencv.org/3.0-beta/modules/tracking/doc/tracker_algorithms.html https://docs.opencv.org/3.0-beta/modules/tracking/doc/tracker_algorithms.html

and

https://docs.opencv.org/3.0-beta/modules/tracking/doc/tracking.html https://docs.opencv.org/3.0-beta/modules/tracking/doc/tracking.html

Westa

[AZhu6]

Really thank you for your advice!

I will try that!

[MartyG]

An old RealSense trick for tracking body parts that do not have tracking support is to use the nearest trackable part (such as the hand palm) and then measure what is happening at an offset position from that point. For example, the middle finger could be measured as a point about Y = 7 cm from the center of the palm. The index, ring and pinky fingers would be Y = 7 cm and + or - 'x' cm from the palm centerpoint.

You could also measure both lower and upper areas of the fingers if the upper areas were Y = 7 cm and the lower areas were Y = 3.5 cm.

In the old Windows RealSense SDK (supported up til the '2016 R3' version), the hand module had a map of joints that the camera could recognize and track.

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v2016r3/documentation/html/index.html?jointtype_pxchanddata.html Intel® RealSense™ SDK 2016 R3 Documentation