- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
How to get the first inference latency from benchmark app ?
and also option to run in mini batch to get the mini batch throughput ?
Thanks,
Ajay
Link Copied
1 Reply
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Ajay,
Reported latency value is calculated as median value of all collected latencies. Reported throughput value is a derivative from reported latency and additionally depends on batch size.
To get latency and throughput for respective batches please make appropriate changes to the infer method in the benchmark_app.py
Also, you may refer to Benchmark Python* Tool for more information.
Best regards,
Surya

Reply
Topic Options
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page