Intel® Distribution of OpenVINO™ Toolkit
Community support and discussions about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all things computer vision-related on Intel® platforms.
5932 Discussions

How to get the first inference latency from benchmark app

Ajay_P_Intel
Employee
241 Views

Hi,

How to get the first inference latency from benchmark app ?

 

and also option to run in mini batch to get the mini batch throughput ?

 

Thanks,

Ajay 

 

0 Kudos
1 Reply
SuryaPSC_Intel
Employee
241 Views

Hi Ajay,

Reported latency value is calculated as median value of all collected latencies. Reported throughput value is a derivative from reported latency and additionally depends on batch size.

To get latency and throughput for respective batches please make appropriate changes to the infer method in the benchmark_app.py

Also, you may refer to Benchmark Python* Tool for more information.

Best regards,

Surya

Reply