OpenVINO CPU FP32 & CPU FP16 Performance

[Anjaneya_Srujit_Ram]

Hi,

I had downloaded the latest release of OpenVINO (2019 R2). I am running the "classification_sample" demo python sample present in 2019.1.1 version in 2019 R2 environment with FP32 and FP16 precisions on CPU. 

I am observing the same throughput performance with FP16 and FP32 formats on CPU. Is there any modifications required to improve performance while running the model with FP16 precision. 

Thank you.

[Shubha_R_Intel]

Dear Ramachandruni, Anjaneya Srujit,

It's not advise-able to draw conclusions based on one sample and one model. Whether FP16 on CPU would make a big difference or not is based on several factors, one of which is the model itself. For instance, heavily pipe-lined models (for instance classification followed by Object Detection) would observe greater performance gain than a simple model. I encourage you to perform some experiments with the  benchmark_app and learn about the different performance knobs available to you. Also the OpenVino Performance Topics Document should be valuable to you as well.

Thanks !

Shubha