Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Ajay_P_Intel
Employee
128 Views

How to get the first inference latency from benchmark app

Hi,

How to get the first inference latency from benchmark app ?

 

and also option to run in mini batch to get the mini batch throughput ?

 

Thanks,

Ajay 

 

0 Kudos
1 Reply
SuryaPSC_Intel
Employee
128 Views

Hi Ajay,

Reported latency value is calculated as median value of all collected latencies. Reported throughput value is a derivative from reported latency and additionally depends on batch size.

To get latency and throughput for respective batches please make appropriate changes to the infer method in the benchmark_app.py

Also, you may refer to Benchmark Python* Tool for more information.

Best regards,

Surya

Reply