Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.

Async Inferrence Using Callbacks

FARSHAD_A_Intel
Employee
1,391 Views

I am trying to use the content in Inference Engine sample code directory to implement a callback function for my inference plugin. the intent is to generate async inference requests and capture the results when available. here is the code I am using:

   async_infer_request->SetCompletionCallback(
       [&] {
             tmr.reset();

              async_infer_request->StartAsync();
              std::cout << "  Frame: "<< *infer_req_cnt << "  Inference Completed" << std::endl;
             });

 

when above code runs, the StartAsync() call never gets executed. I tried to step into the DLDT code but there does not seem to be any indication that the call is going through at any time. Was wondering if there is a more intuitive way to accomplish this? Or something is wrong with this implementation. By the way, above is a simplified version of similar function implemented in Inference Engine sample codes.

0 Kudos
5 Replies
Shubha_R_Intel
Employee
1,391 Views

Dear Farshad,

In the Integrate Inference Engine doc I see the following:

infer_request->StartAsync();
infer_request.Wait(IInferRequest::WaitMode::RESULT_READY);

Perhaps you forgot the Wait statement ?

Thanks,

Shubha

0 Kudos
FARSHAD_A_Intel
Employee
1,391 Views

Interestingly the IDZ admin does not forward replies to proper target thread! was wondering if Shubha was ignoring me :)

Thanks

Farshad.

 

From: Akhbari, Farshad
Sent: Thursday, May 16, 2019 12:59 PM
To: IDZ Admin <idz.admin@intel.com>
Subject: RE: New comment for Computer Vision Forum topic: Async Inferrence Using Callbacks

 

Hi Shubha,

 

I have been told the Wait statement blocks the callback and is contradictory to the purpose of callback. In fact, I have been told use of Wait after callback puts the infer engine to INFER_NOT_STARTED state forever. According to DLDT people, if callback is used, it should not be blocked by wait.

 

Anything else may strike you weird?

 

Thanks

Farshad.

0 Kudos
Shubha_R_Intel
Employee
1,391 Views

Dear Farshad,

Have you done SetCompletionCallback() ?

https://docs.openvinotoolkit.org/latest/classInferenceEngine_1_1InferRequest.html

Can you kindly study the sample inference_engine\samples\perfcheck\main.cpp ?

I think it accomplishes what you want to accomplish.

Thanks,

Shubha

 

 

 

0 Kudos
FARSHAD_A_Intel
Employee
1,391 Views

Hi Shubha,

SetCompletionCallback is the Callback function I am trying to troubleshoot. it is the first line of code in original post. I am not sure how this is different than the one in perfcheck sample. In the perfcheck sample, the Callback is called for an array of requests and mine is only for one as they come along.

thanks,

Farshad.

0 Kudos
Shubha_R_Intel
Employee
1,391 Views

Dear Farshad, 

Please encapsulate your code into a main.cpp and send it to me via intel email. Please make it a simple sample-like main.cpp (not too complicated) which demonstrates the problem.

It could very well be a bug.

Thanks,

Shubha

0 Kudos
Reply