Skeleton tracking as fast as kinect2 ?

[JLeme1]

Hello everyone,

I'm working with kinect2 and trying to find a faster and more responsive head + skeleton tracking.

See examples: https://joanielemercier.com/no-logram/ https://joanielemercier.com/no-logram/

I wonder if the Intel devices and SDK have performances close or better than Kinect ? I saw 'up to 90fps' but I doubt this applies to tracking.

Do you have any experience / advice ?

Best

[MartyG]

The best skeletal tracking solution available for the current 400 Series generation of RealSense cameras is the Nuitrack SDK.

https://www.youtube.com/watch?v=gMPtV4NXtUo Intel RealSense D415/D435 and Nuitrack skeletal tracking SDK replace Kinect SDK - YouTube

The default FPS speed on Nuitrack for the 400 Series is apparently 30 FPS, though a user forced it to run at 60 FPS in the configuration file.

https://community.nuitrack.com/t/intel-d435-change-frame-rate/533/4 https://community.nuitrack.com/t/intel-d435-change-frame-rate/533/4

[JLeme1]

Thanks @MartyG, super useful !

One person seemed interested in 60+ skeleton tracking but I couldn't find information about the results.

I wonder if maybe someone from intel who could confirm the 60 - 90fps specs (latency, max resolution) ?

If these specs are confirmed with skeleton tracking it could be a game changer in digital arts.. exemples

Anamorphosis: https://vimeo.com/223640605 https://vimeo.com/223640605

Interactive projection: https://vimeo.com/207528533 https://vimeo.com/207528533

Cc the_jesus Bob_Duffy

Best

[MartyG]

I'm currently working on skeletal tracking myself in the Unity game engine to control every part of a full-body avatar. I built a system that hooks into my custom real-time animation system so that body parts can be moved simply by changing the XYZ coordinates of a simple block between a '0' and '1' range with motion camera inputs.

Here's an old test video from earlier in my project that illustrates the principle of RealSense body joint control in real-time. The tech is much more advanced now, but it'll give you an idea of what it can do.

https://www.youtube.com/watch?v=T_V8SNwLTrY 'My Father's Face' Tech Trailer 7 - YouTube

Whilst it would be easy to control the coordinates with the old '2016 R2' SDK and an SR300 camera (I've done this in the past), I'm trying to develop a camera input system that works with the current generation of RealSense cameras and their driver files. So Nuitrack and its skeletal tracking in Unity is an interesting possible backup option for me if I don't manage to write my own joint tracker soon.

If you are seeking the supported resolutions at 60 and 90 FPS for the 400 Series cameras, the table below from the camera's data sheet document shows the supported modes for those FPS speeds.

In general, the higher the resolution and the higher the FPS, the more data bandwidth that passes through the USB cable.