How to get the first inference latency from benchmark app

[Ajay_P_Intel]

Hi,

How to get the first inference latency from benchmark app ?

and also option to run in mini batch to get the mini batch throughput ?

Thanks,

Ajay 

[SuryaPSC_Intel]

Hi Ajay,

Reported latency value is calculated as median value of all collected latencies. Reported throughput value is a derivative from reported latency and additionally depends on batch size.

To get latency and throughput for respective batches please make appropriate changes to the infer method in the benchmark_app.py

Also, you may refer to Benchmark Python* Tool for more information.

Best regards,

Surya