Community
cancel
Showing results for 
Search instead for 
Did you mean: 
gdioni
New Contributor I
80 Views

Benchmark app default values

Jump to solution

I'm using the benchmark app to find the best configuration for my two models: YoloV3 and Unet, both trained on custom data sets.

I ran two versions of the async configuration, one has -nthreads 4, the other setting has it not specified.

Here's an example, it's ranked from best(top) to worst(bottom) throughput.

Unet_sv_throughput.PNG

I'm trying to wrap my head around this result on why the asynchronous 4 threads are getting beat by the synchronous calls.

0 Kudos

Accepted Solutions
IntelSupport
Community Manager
64 Views

 

Hi gdioni,

 

Thanks for reaching out.

The throughput value depends on the batch size.

When you run the application in the synchronous mode, it creates one infer request and executes the infer method. If you run the application in the asynchronous mode, it creates as many infer requests. The asynchronous approach running multiple inferences in a parallel pipeline. That why the throughput is higher than the synchronous approach.

Check out the following OpenVINO documentation for more information.

https://docs.openvinotoolkit.org/latest/openvino_docs_IE_DG_Integrate_with_customer_application_new_...

 

Regards,

Aznie


View solution in original post

2 Replies
IntelSupport
Community Manager
65 Views

 

Hi gdioni,

 

Thanks for reaching out.

The throughput value depends on the batch size.

When you run the application in the synchronous mode, it creates one infer request and executes the infer method. If you run the application in the asynchronous mode, it creates as many infer requests. The asynchronous approach running multiple inferences in a parallel pipeline. That why the throughput is higher than the synchronous approach.

Check out the following OpenVINO documentation for more information.

https://docs.openvinotoolkit.org/latest/openvino_docs_IE_DG_Integrate_with_customer_application_new_...

 

Regards,

Aznie


View solution in original post

IntelSupport
Community Manager
38 Views

Hi gdioni,

This thread will no longer be monitored since this issue has been resolved. If you need any additional information from Intel, please submit a new question.


Regards,

Aznie