Intel® Distribution of OpenVINO™ Toolkit
Community support and discussions about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all things computer vision-related on Intel® platforms.

Benchmark app default values

gdioni
New Contributor I
201 Views

I'm using the benchmark app to find the best configuration for my two models: YoloV3 and Unet, both trained on custom data sets.

I ran two versions of the async configuration, one has -nthreads 4, the other setting has it not specified.

Here's an example, it's ranked from best(top) to worst(bottom) throughput.

Unet_sv_throughput.PNG

I'm trying to wrap my head around this result on why the asynchronous 4 threads are getting beat by the synchronous calls.

0 Kudos
1 Solution
IntelSupport
Community Manager
185 Views

 

Hi gdioni,

 

Thanks for reaching out.

The throughput value depends on the batch size.

When you run the application in the synchronous mode, it creates one infer request and executes the infer method. If you run the application in the asynchronous mode, it creates as many infer requests. The asynchronous approach running multiple inferences in a parallel pipeline. That why the throughput is higher than the synchronous approach.

Check out the following OpenVINO documentation for more information.

https://docs.openvinotoolkit.org/latest/openvino_docs_IE_DG_Integrate_with_customer_application_new_...

 

Regards,

Aznie


View solution in original post

2 Replies
IntelSupport
Community Manager
186 Views

 

Hi gdioni,

 

Thanks for reaching out.

The throughput value depends on the batch size.

When you run the application in the synchronous mode, it creates one infer request and executes the infer method. If you run the application in the asynchronous mode, it creates as many infer requests. The asynchronous approach running multiple inferences in a parallel pipeline. That why the throughput is higher than the synchronous approach.

Check out the following OpenVINO documentation for more information.

https://docs.openvinotoolkit.org/latest/openvino_docs_IE_DG_Integrate_with_customer_application_new_...

 

Regards,

Aznie


IntelSupport
Community Manager
159 Views

Hi gdioni,

This thread will no longer be monitored since this issue has been resolved. If you need any additional information from Intel, please submit a new question.


Regards,

Aznie



Reply