Unity Toolkit Facetracking - low performance

[CLeuz]

I am using the RealSense SR300, RealSense SDK R2 and the UnityToolkit with the example ARMirror.

I have experienced that the face tracking in Unity is both less stable and often misaligned compared to the C# or C++ samples.

Is the face tracking module different? Or is there a problem with the projection of the tracked landmarks and the camera plane?

[MartyG]

I am a full time Unity developer myself. I think the difference between the Unity and C++ / C# implementations is probably because the PC has the extra processing demand of running the full Unity environment, whereas a C# / C++ program is much less demanding on computer resources and so there is less potential for processing lag. Also, the way that the AR Mirror sample is designed - taking a loose pile of pieces that are not attached to anything and having them leap up to the tracked point - may have an impact on alignment.

[CLeuz]

Thanks for your answer!

Since ARToolkit or OpenCV have tracking solutions for Unity that align perfectly, I thought that this might be a problem of the alignment, FOV,... between main camera and AR camera.

[MartyG]

I have used RealSense with Unity for years. My experience is that overall it behaves excellently, and any glitches that occur are usually the consequence of the coding design in a particular program and not a problem with the camera hardware.

As you are using the '2016 R2' SDK, you may be interested in my large range of step by step guides to using various aspects of RealSense with Unity in that SDK.

https://software.intel.com/en-us/forums/realsense/topic/676139 Index of Marty G's RealSense Unity How-To Guides

[CLeuz]

Thanks a lot for sharing these guides!