8-bit integer precesion

[Stefano_M_]

From release note I read that:

"Adds low precision, 8-bit integer (Int8) inference as a preview feature for Intel CPUs to achieve optimized runs. A calibration tool with built-in samples saves calibrated intermediate representation (IR) files with embedded statistics on the Int8 profile."

How can I convert (with Model Optimizer) my trained tensor flow model using an 8-bit integer precision?

Is FP16 supportend for inference using CPU?

Thanks,

Stefano

[Andrey_M_Intel]

You need to follow "Low-Precision 8-bit Integer Inference" topic in in-package documentation.

Shortly, you have to use calibration tool which accept FP32 IR, calibration data set, will calibrate model and produce calibrated IR which will be able to be loaded to CPU plugin and executed in INT8 on platforms supporting AVX-512

[Stefano_M_]

Thank you, I will try.

Is FP16 supported on CPU?

[nikos1]

Andrey Malyshev (Intel) wrote:

You need to follow "Low-Precision 8-bit Integer Inference" topic in in-package documentation.

Shortly, you have to use calibration tool which accept FP32 IR, calibration data set, will calibrate model and produce calibrated IR which will be able to be loaded to CPU plugin and executed in INT8 on platforms supporting AVX-512

Any guidelines on the size of the calibration data set?  What are typical sizes used? 100 , 1000 , 10K ?

Never mind.. just found this - attaching for future reference.

..

The calibration dataset contains images and can be a subset of the validation set. A small fraction of images from validation dataset (1-5%) is enough to create a calibration dataset.

[nikos1]

Stefano M. wrote:

Thank you, I will try.

Is FP16 supported on CPU?

> Is FP16 supported on CPU?

I believe not supported on CPU according to the documentation.  OpenVino team can confirm.

[Stefano_M_]

ok thanks