Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6555 Discussions

How to get the first inference latency from benchmark app

Ajay_P_Intel
Employee
‎05-20-2020 01:42 AM
1,094 Views

Hi,

How to get the first inference latency from benchmark app ?

 

and also option to run in mini batch to get the mini batch throughput ?

 

Thanks,

Ajay 

 

0 Kudos

Link Copied

1 Reply
SuryaPSC_Intel
Employee
‎05-20-2020 02:22 AM
1,094 Views

Hi Ajay,

Reported latency value is calculated as median value of all collected latencies. Reported throughput value is a derivative from reported latency and additionally depends on batch size.

To get latency and throughput for respective batches please make appropriate changes to the infer method in the benchmark_app.py

Also, you may refer to Benchmark Python* Tool for more information.

Best regards,

Surya

0 Kudos
Copy link
Reply

Community support is provided Monday to Friday. Other contact methods are available here.

Intel does not verify all solutions, including but not limited to any file transfers that may appear in this community. Accordingly, Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.

For more complete information about compiler optimizations, see our Optimization Notice.