Bug: face tracking jumping when not in ifall mode

[Zsolt_E_]

The face tracking algorithm is jumping like crazy, 3-4 times per second if, in AcquireFrame(false) mode.

When in ifall mode, it's nice and smooth. 

This is the code I'm using:

face_output_data->Update();
        int numfaces = face_output_data->QueryNumberOfDetectedFaces();
        if (numfaces > 0)
        {
            PXCFaceData::Face* face = face_output_data->QueryFaceByIndex(0);
            if (face != 0)
            {
                PXCFaceData::DetectionData* ddata = face->QueryDetection();
                if (ddata != 0)
                {
                    PXCRectI32 rect;
                    if (ddata->QueryBoundingRect(&rect))
                    {
                        face_pos.x = rect.x + rect.w / 2;
                        face_pos.y = rect.y + rect.h / 2;
                    }
                }
            }
        }

[Xusheng_L_Intel]

It is not a bug. You need use AcquireFrame(true) mode since we need all processing modules complete generating results. Thanks!

[Zsolt_E_]

David, I'll try to make a video of it, it really does "feel" like a bug. If you'd see the video you'd understand why I'm saying it.

In 90-95% of the frames the position is correct, in the rest it's offset by about 100,100 pixels to the top left.

If you say that face detection should only work in the AcquireFrame(true) mode, you are saying that you are limiting the framerate of the game to 15 fps, the auto-exposure limit on the RGB stream in lower light conditions. Having 15 fps for any kind of computer interaction is very bad UX.

At the moment the only way to use the SDK with face tracking + depth + color is in Streaming mode, which is a really advanced programming task to synchronise with a client. 

[Rupam_D_]

This happens due to light flickering and not a bug. Even OpenCV has this problem. This is more prominent under artificial light like Tube light.

You can use median filter to get rid of this problem.

Alternatively you can store past two values and measure the distance of current face rectangle top, if the deviation is very high, instead of current face coordinates, take the previous coordinate. I have done quite a bit of work with face detection, and therefore can safely tell you there is no other alternative. RealSense's smoothing filter ( 3D with z=0) is also very good and fast. It is worth a try ( 3 point filter). 

[Zsolt_E_]

Rupam, it works perfectly under streaming mode! It is simply a bug with the AcquireFrame(false) mode.