Speed gain after INT8 conversion

[Sen_Gupta__Indrajit]

Hi All,

I converted a MobileNetV2 model into IR format and calibrated it to create the INT8 version of the model. The FPS that I got on the original model using the benchmark tool was 51.16. On the converted INT8 model it comes out to be 51.73. My baseline is expected to be low since I am running this on my laptop - but the speed gain is only about 1.01x which is hardly anything.

Any thoughts on what can be done to improve the FPS?

Indrajit

[Shubha_R_Intel]

Dear Sen Gupta, Indrajit,

You ask an interesting question. What I recommend that you do is to use the benchmark_app to experiment with different parameters and find out for yourself. There are a few knobs you can turn to run your experiments such as CPU-thread-pinning, number of threads to use on the CPU, batch_size, async API etc...

Please post here should you have further questions.

Thanks,

Shubha