INTEL® REALSENSE™ 3D CAMERA FOR SIGN LANGUAGE FINGER MOTION

[Tony_S_1]

Hi - Newbie here.  I was directed to this forum by the folks at Intel Higher Education. I have a question about the use of Intel Realsense 3D Cameras and software for Sign Language research. I am a research student working on finger sign language recognition. In particular, I need to measure speed of fingers as well as amount of movement in sign language. I came across Intel realsense and was wondering if it has the software to process large volumes of recorded sign language gestures in order to automatically determine speed and range of finger motion.  I am not a programmer, so ideally there would be a non-technical solution for this. Any advice you could provide would be really appreciate. Thanks. Tony

[Felipe_P_Intel]

Hi Tony,

There is a Brazilian developer/researcher that already developed a solution to recognize Sign Language using RealSense: https://www.youtube.com/watch?v=Dj8Ut1REYhY

His name is Gabriel Moreira and he published a paper describing the techniques that he used: http://rd.springer.com/chapter/10.1007%2F978-3-319-12027-0_51

Talking about the speed of joints, there are some settings that you can do to configure that: https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/manuals_configuring_tracking_options.html

Regards,

Felipe

[samontab]

Hi Tony,

There isn't a ready to use solution for this.

The raw data acquisition from the camera seems to be fast enough (more than 60Hz) for detecting that type of movement, however some clever processing needs to be done with that data, which may not be as fast.

This is a rather challenging application but I think it can be done to a certain degree.

[Anton_K_2]

GestuReader application - a software for American Sign Language interpreting using computer 3D vision. Through the use of Intel RealSense technology GestuReader removes communication barriers between people with different abilities.

www.gestureader.com

GestuReader features:

• recognition of all 36 signs (A-Z, 0-9) of the ASL alphabet;
• there is no need to wear gloves or use other devices; signs are made with one’s own bare hands;
• performing various actions depending on recognition result:
  • either text entry (with auto-completion) in the current entry field,
  • or sending messages (key combinations) to the current window (e.g. PageUp, Left, Right, Ctrl-Insert, etc.),
  • or launching other applications;
• transforming speech into text and vice versa in various languages;
• support for all front-facing Intel RealSense cameras (tested on F200 and SR300);
• multilingual user interface and documentation: English, French, German, Italian, Spanish, Russian, Chinese (Traditional);
• ability to use both hands at the same time.