problem with simulatneous using gestures and face tracking

[Piotr_P_]

Hi all,

I have a problem with using of several modules simultaneously. I am trying to use gestures, touchless controler, and face tracking algorithms. But when I enable face tracking module I can't use gestures ("camera doesn't see them"). I am using WPF C#. I'm quite sure that I am doing something wrong with using SDK with several modules. Below is my code.

	public MainWindow()
        {
            InitializeComponent();

            InitRealSense();
            InitTouchlessController();
            InitGestures();
        }

        private void Window_Closing(object sender, System.ComponentModel.CancelEventArgs e)
        {
            DisposeRealSense();
        }

        public void InitRealSense()
        {
            // Create a SenseManager instance.
            sm = PXCMSenseManager.CreateInstance();
            // Enable the touchless controller feature.
            sm.EnableTouchlessController();
            // Enable hand tracking
            sm.EnableHand();
	    // Enable face tracking
            sm.EnableFace();
            // Get a face instance here for configuration
            PXCMFaceModule face = sm.QueryFace();
            // Initialize
            PXCMSenseManager.Handler handler = new PXCMSenseManager.Handler
            {
                onModuleProcessedFrame = OnModuleProcessedFrame
            };

            sm.Init(handler);

            // Get a hand instance here for configuration
            hand = sm.QueryHand();
            hcfg = hand.CreateActiveConfiguration();
        }

        pxcmStatus OnModuleProcessedFrame(int mid, PXCMBase module, PXCMCapture.Sample sample)
        {
            // check if the callback is from the face tracking module.
            if (mid == PXCMFaceModule.CUID)
            {
                PXCMFaceModule face = module.QueryInstance<PXCMFaceModule>();
		// do something
            }
            // return NO_ERROR to continue, or any error to abort.
            return pxcmStatus.PXCM_STATUS_NO_ERROR;
        }


        public void InitGestures()
        {
            hcfg.DisableAllGestures();
            hcfg.EnableGesture("wave");
            hcfg.SubscribeGesture(OnFiredGesture);
            hcfg.ApplyChanges();
        }

        void OnFiredGesture(PXCMHandData.GestureData data) {
            // do something
        }

        public void InitTouchlessController() {
            // Get the module instance 
            PXCMTouchlessController tc = sm.QueryTouchlessController();
            // register for the ux events
            tc.SubscribeEvent(OnFiredUXEvent);

            // start processing thread
            processingThread = new Thread(new ThreadStart(ProcessingThread));
            processingThread.SetApartmentState(ApartmentState.STA);
            processingThread.Start();

        }

        public void DisposeRealSense() {
            hand.Dispose();
            hcfg.Dispose();
            // Clean up
            processingThread.Abort();
            // close session manager
            sm.Dispose();
            
        }

        private void ProcessingThread()
        {
            // Streaming data
            while (sm.AcquireFrame(true) >= pxcmStatus.PXCM_STATUS_NO_ERROR)
            {
                sm.ReleaseFrame();
            }
        }
        

        

Any idea why this is not working or how to properly use several modules enabled?

Thanks. Piotr

[Nirmit_Kavaiya]

I am facing this same issue. Hands are detected, but gestures are not.

[Wezley_S_ · Accepted solution]

Hi Piotr,

I managed to get around this by creating new classes for the specific module and running them on different threads.

When I did this I had to make sure that only one enabled at a time (Don't enable them at all once! Wait for once to initialize then have it call on another to initialize and so on) or else you'll end up getting an access violation.

If someone has an easier way to do this please comment!

[Piotr_P_]

Thanks Wezley, I think that each module should be a separate class by design! If I find an easier way I will write about it. Once again thanks.

[Wezley_S_]

No problem.

One big problem I've come across though is that when multiple modules are active there is a huge impact on performance. Even when they're run on separate threads and sessions. Which is quite annoying.

I created a new thread on that and will hopefully be presented with a fix soon.