Async Inferrence Using Callbacks

[FARSHAD_A_Intel]

I am trying to use the content in Inference Engine sample code directory to implement a callback function for my inference plugin. the intent is to generate async inference requests and capture the results when available. here is the code I am using:

   async_infer_request->SetCompletionCallback(
       [&] {
             tmr.reset();

              async_infer_request->StartAsync();
              std::cout << "  Frame: "<< *infer_req_cnt << "  Inference Completed" << std::endl;
             });

when above code runs, the StartAsync() call never gets executed. I tried to step into the DLDT code but there does not seem to be any indication that the call is going through at any time. Was wondering if there is a more intuitive way to accomplish this? Or something is wrong with this implementation. By the way, above is a simplified version of similar function implemented in Inference Engine sample codes.

[Shubha_R_Intel]

Dear Farshad,

In the Integrate Inference Engine doc I see the following:

infer_request->StartAsync();
infer_request.Wait(IInferRequest::WaitMode::RESULT_READY);

Perhaps you forgot the Wait statement ?

Thanks,

Shubha

[FARSHAD_A_Intel]

Interestingly the IDZ admin does not forward replies to proper target thread! was wondering if Shubha was ignoring me :)

Thanks

Farshad.

From: Akhbari, Farshad
Sent: Thursday, May 16, 2019 12:59 PM
To: IDZ Admin <idz.admin@intel.com>
Subject: RE: New comment for Computer Vision Forum topic: Async Inferrence Using Callbacks

Hi Shubha,

I have been told the Wait statement blocks the callback and is contradictory to the purpose of callback. In fact, I have been told use of Wait after callback puts the infer engine to INFER_NOT_STARTED state forever. According to DLDT people, if callback is used, it should not be blocked by wait.

Anything else may strike you weird?

Thanks

Farshad.

[Shubha_R_Intel]

Dear Farshad,

Have you done SetCompletionCallback() ?

https://docs.openvinotoolkit.org/latest/classInferenceEngine_1_1InferRequest.html

Can you kindly study the sample inference_engine\samples\perfcheck\main.cpp ?

I think it accomplishes what you want to accomplish.

Thanks,

Shubha

[FARSHAD_A_Intel]

Hi Shubha,

SetCompletionCallback is the Callback function I am trying to troubleshoot. it is the first line of code in original post. I am not sure how this is different than the one in perfcheck sample. In the perfcheck sample, the Callback is called for an array of requests and mine is only for one as they come along.

thanks,

Farshad.

[Shubha_R_Intel]

Dear Farshad, 

Please encapsulate your code into a main.cpp and send it to me via intel email. Please make it a simple sample-like main.cpp (not too complicated) which demonstrates the problem.

It could very well be a bug.

Thanks,

Shubha