Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.

Benchmark app default values

gdioni
New Contributor I
422 Views

I'm using the benchmark app to find the best configuration for my two models: YoloV3 and Unet, both trained on custom data sets.

I ran two versions of the async configuration, one has -nthreads 4, the other setting has it not specified.

Here's an example, it's ranked from best(top) to worst(bottom) throughput.

Unet_sv_throughput.PNG

I'm trying to wrap my head around this result on why the asynchronous 4 threads are getting beat by the synchronous calls.

0 Kudos
1 Solution
IntelSupport
Community Manager
406 Views

 

Hi gdioni,

 

Thanks for reaching out.

The throughput value depends on the batch size.

When you run the application in the synchronous mode, it creates one infer request and executes the infer method. If you run the application in the asynchronous mode, it creates as many infer requests. The asynchronous approach running multiple inferences in a parallel pipeline. That why the throughput is higher than the synchronous approach.

Check out the following OpenVINO documentation for more information.

https://docs.openvinotoolkit.org/latest/openvino_docs_IE_DG_Integrate_with_customer_application_new_API.html

 

Regards,

Aznie


View solution in original post

0 Kudos
2 Replies
IntelSupport
Community Manager
407 Views

 

Hi gdioni,

 

Thanks for reaching out.

The throughput value depends on the batch size.

When you run the application in the synchronous mode, it creates one infer request and executes the infer method. If you run the application in the asynchronous mode, it creates as many infer requests. The asynchronous approach running multiple inferences in a parallel pipeline. That why the throughput is higher than the synchronous approach.

Check out the following OpenVINO documentation for more information.

https://docs.openvinotoolkit.org/latest/openvino_docs_IE_DG_Integrate_with_customer_application_new_API.html

 

Regards,

Aznie


0 Kudos
IntelSupport
Community Manager
380 Views

Hi gdioni,

This thread will no longer be monitored since this issue has been resolved. If you need any additional information from Intel, please submit a new question.


Regards,

Aznie



0 Kudos
Reply