Using multiple Face Recognition databases

[GProf]

Hi,

Based on @David Lu's reply from this topic I'm trying to get past the 33 faces per database limitation by loading multiple databases.

Unfortunately I get a System.AccessViolationException in libpxcclr.cs.dll using the C# API when I try to call

//recognitionConfig is a RecognitionConfiguration instance, and databases is a Byte[][] containing multiple databases already loaded
recognitionConfig.SetDatabaseBuffer(databases[currentDatabaseIndex]);

I did setup a very basic state machine to make sure this method is not called while other RSSDK queries are happening, 
but I still get a crash.

Is it possible to swap between multiple Face Recognition buffers at runtime ?
If so, now ? If not, what other alternatives do you suggest for using a larger number of faces for recognition ?

Thank you,
George

[GProf]

I've tried a couple of different ways of swapping face recognition databases, the most crude being releasing the whole RSSDK pipeline and re-initializing it.

After having a deeper look in the documentation, re-creating a RecognitionConfiguration works, although applying the changes takes a bit of time:

senseManager.PauseModule(PXCMFaceModule.CUID, true);


            PXCMFaceModule faceModule;
            PXCMFaceConfiguration faceConfig;


            faceModule = senseManager.QueryFace();

            faceConfig = faceModule.CreateActiveConfiguration();

            recognitionConfig.Disable();
            recognitionConfig = null;
            recognitionConfig = faceConfig.QueryRecognition();

            //recognitionConfig = recognitionConfigs[currentDatabaseIndex];

            recognitionConfig.Enable();

            // Create a recognition database
            PXCMFaceConfiguration.RecognitionConfiguration.RecognitionStorageDesc recognitionDesc = new PXCMFaceConfiguration.RecognitionConfiguration.RecognitionStorageDesc();
            recognitionDesc.maxUsers = DatabaseUsers;
            recognitionConfig.CreateStorage(databaseNames[currentDatabaseIndex], out recognitionDesc);
            recognitionConfig.UseStorage(databaseNames[currentDatabaseIndex]);
            recognitionConfig.SetDatabaseBuffer(databases[currentDatabaseIndex]);

            // Apply changes and initialize
            faceConfig.ApplyChanges();

            faceData.Dispose();
            faceData = null;
            faceData = faceModule.CreateOutput();

            // Release resources
            faceConfig.Dispose();
            faceModule.Dispose();

            senseManager.PauseModule(PXCMFaceModule.CUID, false);

I have also tried caching multiple configurations which are disabled initially then later enabled as needed:

senseManager.PauseModule(PXCMFaceModule.CUID, true);

                    PXCMFaceModule faceModule = senseManager.QueryFace(); 
                    PXCMFaceConfiguration faceConfig = faceModule.CreateActiveConfiguration();
                    
                    recognitionConfigs[currentDatabaseIndex].Disable();
                    currentDatabaseIndex = (currentDatabaseIndex + 1) % numDatabases;
                    recognitionConfigs[currentDatabaseIndex].Enable();
                    
                    faceConfig.ApplyChanges();

                    faceData.Dispose();
                    faceData = null;
                    faceData = faceModule.CreateOutput();

                    // Release resources
                    faceConfig.Dispose();
                    faceModule.Dispose();

but this didn't seem to work. Is the above syntax correct ?

What is the fastest recommended way of swapping face recognition databases and configurations ?

[Daniel_M_4]

Hi,

I am also looking into getting past the face per database limitation as well since I plan on dealing with a large amount of faces. I was wondering if you or anyone else on this forum had an update on a good solution for swapping face recognition databases and configurations without sacrificing too much speed when it comes to predictions. Alternatively, if there was a way to increase the face per database limitation, I would be interested in that as well.

Regards,

Daniel

[GProf]

Hi Daniel,

In newer updates the to the SDK the number of faces supported by the face recognition algorithm decrease (perhaps to increase stability/performance at the time time being).

Intel employees can correct me, but if will take some time until the face recognition module will increase the number of faces (if at all)

For production purposes I recommend looking at alternatives.

[MartyG]

I'm not sure how relevant this will be to the conversation, but in general when working with RealSense in Unity I have found that limitations or conflicts can be resolved by creating uniquely named copies of modules and pointing scripts towards them,

For example, in the RealSense motion tracking in Unity, you have a primary tracking script (TrackingAction) and sub-scripts that TrackingAction relies on for various variables (HandTrackingRule, FaceTrackingRule, HandDetectedRule, etc).  I was able to make two-handed control more stable by creating a second set of the scripts (TrackingAction2, HandTrackingRule2, FaceTrackingRule2, HandDetectedRule2).  

I edited the TrackingAction2 to point to the '2' versions of the support scripts instead of the originals.  By doing so, I was redirecting processing of inputs related to the opposite hand to this second set instead of a single set of scripts trying to process both hands at the same time.  Control of the second hand therefore became as smooth as the first hand's controls, because it was like the two hands were in alternate universes, present but also apart and not interacting with one another.

I don't know if you can make anything useful for dealing with your face database problem from that information.  Just thought I'd put my two cents worth into the discussion.  :)

[Daniel_M_4]

Alright, thanks for the input, guys! I definitely appreciate it. I suppose I may have to consider an alternative unless anyone else here knows an effective way or at least a working solution to deal with a large amount of faces with Intel Realsense...

[Raja_R_]

Hi Daniel,

Daniel M. wrote:

Alright, thanks for the input, guys! I definitely appreciate it. I suppose I may have to consider an alternative unless anyone else here knows an effective way or at least a working solution to deal with a large amount of faces with Intel Realsense...

Did you manage to solve this issue? If so how? 

Many thanks

R

[Daniel_M_4]

No, unfortunately I was not able to solve this issue. I just ended up settling for working within the constraint.

[Raja_R_]

Thanks for getting back to me :)

Does this mean that the system you developed will only work with up to 33 faces?

TIA

R

[Daniel_M_4]

Well if I remember correctly, they have actually decreased the facial database limit down to 20 faces. But yes, the system I have developed cannot go past the limit as is.

Regards,

Daniel