Information about emotion recognition via Intel RealSense

[ETara]

Hello! I am writing a work about emotion recognition using Intel RealSense. Can I, please, get some extra information about it in the context of emotion recognition algorithm (like how does it recognize the face, overlay a template with special points on the face and etc.). Thank you for your time!

[MartyG · Accepted solution]

RealSense did have an emotion recognition algorithm up until a couple of years ago but it is sadly no longer part of the RealSense SDK, because it was a third-party component by a company called Emotient that was purchased by Apple, and presumably the licence to use the technology was not renewed for this reason.

There is another way to detect emotion with RealSense, though it requires a bit more labor to get up and running than the old Emotient module did. I detailed the procedure I use in my own project in the comment linked to below.

[ETara]

Thank you for your answer! I understand that you didn't use landmark information in your system, but can you please explain how camera could get landmark information via RealSense SDK (what is the algorithm to get the values).

[MartyG]

You can get information from specific landmark points on the face with the QueryPoint instruction.

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?querypoint_landmarksdata_pxcfacedata.html Intel® RealSense™ SDK 2016 R2 Documentation

The values of particular landmark points are on a face chart.

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?doc_face_face_landmark_data.html Intel® RealSense™ SDK 2016 R2 Documentation