How to combine PXCBlobExtractor and PXCSenseManager

[Zsolt_E_]

I'm trying to make an app which features both PXCBlobExtractor and face tracking. 
I am confused, because the recommended "normal" way to use the SDK seems to be along the lines of:

PXCSenseManager *psm=PXCSenseManager::CreateInstance();
psm->EnableStream(PXCCapture::STREAM_TYPE_DEPTH, 640, 480, 60);
...
{
    if (psm->AcquireFrame(true)<PXC_STATUS_NO_ERROR) break; 
    PXCCapture::Sample *sample = psm->QuerySample();
}

however the two examples which feature PXCBlobExtractor (mask_utils and reference manual example 65) use a very different way for PXCBlobExtractor, where a sense manager is created from a session object. I find the structure of the PXCSenseManager::CreateInstance() applications much cleaner and better organizes and I don't yet understand the "advanced" usage like for example the mask_utils example.

Can someone provide a very minimalistic example which shows what is the best way to combine PXCBlobExtractor with normal functions like Face Tracking? Ideally, I'd like to set the camera properties like FilterOption, etc. Can you tell me how can I combine the 3 options in a very minimalistic main function?

[Artem_V_Intel]

Hi Zsolt E.

Please use QuerySession() method of PXCSenseManager interface to get an instance of underlying PXCSession object and then create an implementation of PXCBlobExtractor interface using this session object.

PXCSession* session = psm->QuerySession();
PXCBlobExtractor *be = nullptr;
pxcStatus sts = session->CreateImpl<PXCBlobExtractor>(&be);
// use blob extractor here ...
be->Release();

Keep in mind that you should not call Release() method of a PXCSession object received from QuerySession() method.

Best regards,

Artem

[Zsolt_E_]

Hi Artem,

Thanks for your help. I've managed to make the following structure, based on the best practices found in the SDK manual and the samples.

#include <vector>
#include "stdafx.h"
#include "pxcsensemanager.h"
#include "utilities/pxcmaskutils.h"
#include "windows.h"
#include <iostream>


int _tmain(int argc, _TCHAR* argv[])
{
    PXCSenseManager* psm = PXCSenseManager::CreateInstance();
    if (!psm)
    {
        wprintf_s(L"Error: PXCSenseManager is unavailable\n");
        return -1;
    }
    PXCSession *g_session = psm->QuerySession();

    psm->EnableStream(PXCCapture::STREAM_TYPE_DEPTH, 640, 480, 60);

    if (psm->Init() < PXC_STATUS_NO_ERROR)
    {
        wprintf_s(L"Init error\n");
        return -1;
    }

    PXCCapture::Device* device = psm->QueryCaptureManager()->QueryDevice();
    if (!device)
    {
        wprintf_s(L"Error: QueryDevice failed on line %d\n", __LINE__);
        return -1;
    }

    PXCCapture::Device::PropertyInfo info;
    info = device->QueryDepthConfidenceThresholdInfo();
    device->SetDepthConfidenceThreshold((pxcI16)info.defaultValue);

    device->SetIVCAMLaserPower(16);
    device->SetIVCAMAccuracy(PXCCapture::Device::IVCAM_ACCURACY_FINEST);
    device->SetIVCAMMotionRangeTradeOff(0);
    device->SetIVCAMFilterOption(1);
    device->SetMirrorMode(PXCCapture::Device::MIRROR_MODE_DISABLED);

    while (true) {
        if (psm->AcquireFrame(true) < PXC_STATUS_NO_ERROR) {
            break;
        }
        PXCCapture::Sample *sample = psm->QuerySample();
        
        ExtractBlobs(g_session, ...... HOW TO GET IMAGE);
        
        psm->ReleaseFrame();
    }

    psm->Close();
}

void ExtractBlobs(PXCSession *session, PXCImage *image, std::vector<PXCBlobExtractor::BlobData> &blobs) {
    blobs.clear();

    // Create an instance of the blob extractor
    PXCBlobExtractor *blob = 0;
    pxcStatus sts = session->CreateImpl<PXCBlobExtractor>(&blob);
    if (sts < PXC_STATUS_NO_ERROR) return;

    // Initialize
    PXCImage::ImageInfo iinfo = image->QueryInfo();
    PXCImage *segImage = session->CreateImage(&iinfo);
    blob->Init(iinfo);

    // Extract blobs from the image
    blob->ProcessImage(*image);

    // Prepare blob data to return back
    int nblobs = blob->QueryNumberOfBlobs();
    for (int i = 0; i < nblobs; i++) {
        PXCBlobExtractor::BlobData data2;
        blob->QueryBlobData(i, *segImage, data2);
        blobs.push_back(data2);
    }

    // Clean up
    segImage->Release();
    blob->Release();
}

I have one problem now, I don't know how to get an image which is required for ExtractBlobs. Should I try to get an image from PXCCapture::Sample?

Also, do you think the structure of the above code is OK?