Understanding on -nireq and -nstreams

[Thomas__Sruthi]

Hello,

We have been running the benchmark_app on Intel HD Graphics 630 to see how the GPU can be stressed. The -nireq,-nstreams and -b switches have been tried too. What was understood about -nstreams is that it's the number of threads spawned, from running top. Is that correct?

Please help us with an understanding on what each of these switches signify in the GPU, for better stressing.

Thanks,

Sruthi

[Luis_at_Intel]

Hi Thomas, Sruthi,

Please take a look at the Throughput Mode for CPU section under Introduction to Performance Topics. This should give you a pretty good understanding of what the -nstreams and the other flags do. 

The -nireq is just the number of infer requests, a default value is determined automatically for a device but you can play around with this number and see which gives the best throughput. In addition to the number of streams, it is also possible to play with the batch size (-b flag) to find the throughput sweet-spot. Running multiple independent inference requests in parallel often gives much better performance, than using a batch only.

Regards,

Luis