Gaze Detection using RealSense Camera

[Mona_J_]

Hi,

I am wondering if you might have any (un-)published code sample for gaze detection or if you know of a trick that I can do gaze detection using already known samples codes (as an example using Face Detection)? I would like to know if a person is looking straight or if they are looking down.

Thanks,

Mona Jalal.

[mzc]

I am also looking for the similar thing, any help on the gaze calibration part at least?

Thanks!

[Mona_J_1]

I wonder if gaze detection and gaze direction detection is already implemented in the new version of SDK which was released recently and if so, if there's any demo which can tell us which functions we should use to make use of it?

[Colleen_C_Intel]

It is implemented in v6, release 4. See documentation here https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/manuals_gazetracking_f200.html

[mzc]

Thanks.. yes, I just downloaded the SDK.

Now to calibrate the gaze I need to build a ui where different points indicating the coordinate and match with the codes here. wondering if someone also did that too!

[mzc]

Hi again, I have changed some basic code for gaze tracking, but it is giving some random x,y gaze coordinate.. could you please have a look:

namespace Gazetrack
{
    public partial class MainWindow : Window
    {
        private Thread processingThread;
        private PXCMSenseManager senseManager;

        private PXCMFaceModule face;
        private PXCMFaceConfiguration faceConfiguration;
        private PXCMFaceConfiguration.GazeConfiguration gazec;

        public MainWindow()
        {
            InitializeComponent();

            // Instantiate and initialize the SenseManager
            senseManager = PXCMSenseManager.CreateInstance();
            senseManager.EnableStream(PXCMCapture.StreamType.STREAM_TYPE_COLOR, 640, 480, 60);
            senseManager.EnableFace();
            senseManager.Init();

            ConfigureFaceTracking();

            // Start the worker thread
            processingThread = new Thread(new ThreadStart(ProcessingThread));
            processingThread.Start();
        }

        private void ConfigureFaceTracking()
        {
            face = senseManager.QueryFace();
            faceConfiguration = face.CreateActiveConfiguration();
            faceConfiguration.detection.isEnabled = true;

            faceConfiguration.EnableAllAlerts();

            //Gaze detection
            gazec = faceConfiguration.QueryGaze();
            gazec.isEnabled = true;
            faceConfiguration.ApplyChanges();
        }

        private void Window_Loaded(object sender, RoutedEventArgs e)
        {
            lblMessage.Content = "(Wave Your Hand)";
        }

        private void Window_Closing(object sender, System.ComponentModel.CancelEventArgs e)
        {
            processingThread.Abort();
            senseManager.Dispose();
        }

        private void ProcessingThread()
        {
            // Start AcquireFrame/ReleaseFrame loop
            while (senseManager.AcquireFrame(true) >= pxcmStatus.PXCM_STATUS_NO_ERROR)
            {

                // Get a face instance
                face = senseManager.QueryFace();

                if (face != null)
                {
                    // Get face tracking processed data
                    PXCMFaceData faceData = face.CreateOutput();
                    faceData.Update();

                    Int32 nfaces = faceData.QueryNumberOfDetectedFaces();

                    for (Int32 i = 0; i < nfaces; i++)
                    {
                        // Retrieve the data instance
                        PXCMFaceData.Face face1 = faceData.QueryFaceByIndex(i);
                        PXCMFaceData.GazeData gazed = face1.QueryGaze();

                        if (gazed != null) { 
                        // retrieve the gaze information
                        PXCMFaceData.GazePoint gazep = gazed.QueryGazePoint();
                        double angleh = gazed.QueryGazeHorizontalAngle();
                        double anglev = gazed.QueryGazeVerticalAngle();

                        // action based on detected expression
                        Console.WriteLine("(x,y): " + gazep.screenPoint.x + "," + gazep.screenPoint.y);
                    
                        }
                    }
                }
                senseManager.ReleaseFrame();
            }
        }
    }
}