I converted a MobileNetV2 model into IR format and calibrated it to create the INT8 version of the model. The FPS that I got on the original model using the benchmark tool was 51.16. On the converted INT8 model it comes out to be 51.73. My baseline is expected to be low since I am running this on my laptop - but the speed gain is only about 1.01x which is hardly anything.
Any thoughts on what can be done to improve the FPS?
Dear Sen Gupta, Indrajit,
You ask an interesting question. What I recommend that you do is to use the benchmark_app to experiment with different parameters and find out for yourself. There are a few knobs you can turn to run your experiments such as CPU-thread-pinning, number of threads to use on the CPU, batch_size, async API etc...
Please post here should you have further questions.