Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Beginner
25 Views

8-bit integer precesion

From release note I read that:

"Adds low precision, 8-bit integer (Int8) inference as a preview feature for Intel CPUs to achieve optimized runs. A calibration tool with built-in samples saves calibrated intermediate representation (IR) files with embedded statistics on the Int8 profile."

How can I convert (with Model Optimizer) my trained tensor flow model using an 8-bit integer precision?

Is FP16 supportend for inference using CPU?

Thanks,

Stefano

0 Kudos
5 Replies
Highlighted
Employee
25 Views

You need to follow "Low-Precision 8-bit Integer Inference" topic in in-package documentation.

Shortly, you have to use calibration tool which accept FP32 IR, calibration data set, will calibrate model and produce calibrated IR which will be able to be loaded to CPU plugin and executed in INT8 on platforms supporting AVX-512

0 Kudos
Highlighted
Beginner
25 Views

Thank you, I will try.

Is FP16 supported on CPU?

 

0 Kudos
Highlighted
Valued Contributor I
25 Views

Andrey Malyshev (Intel) wrote:

You need to follow "Low-Precision 8-bit Integer Inference" topic in in-package documentation.

Shortly, you have to use calibration tool which accept FP32 IR, calibration data set, will calibrate model and produce calibrated IR which will be able to be loaded to CPU plugin and executed in INT8 on platforms supporting AVX-512

 

Any guidelines on the size of the calibration data set?  What are typical sizes used? 100 , 1000 , 10K ?

Never mind.. just found this - attaching for future reference.

..

The calibration dataset contains images and can be a subset of the validation set. A small fraction of images from validation dataset (1-5%) is enough to create a calibration dataset.

0 Kudos
Highlighted
Valued Contributor I
25 Views

Stefano M. wrote:

Thank you, I will try.

Is FP16 supported on CPU?

 

 

> Is FP16 supported on CPU?

I believe not supported on CPU according to the documentation.  OpenVino team can confirm.

 

0 Kudos
Highlighted
Beginner
25 Views

ok thanks

0 Kudos