- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
From release note I read that:
"Adds low precision, 8-bit integer (Int8) inference as a preview feature for Intel CPUs to achieve optimized runs. A calibration tool with built-in samples saves calibrated intermediate representation (IR) files with embedded statistics on the Int8 profile."
How can I convert (with Model Optimizer) my trained tensor flow model using an 8-bit integer precision?
Is FP16 supportend for inference using CPU?
Thanks,
Stefano
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You need to follow "Low-Precision 8-bit Integer Inference" topic in in-package documentation.
Shortly, you have to use calibration tool which accept FP32 IR, calibration data set, will calibrate model and produce calibrated IR which will be able to be loaded to CPU plugin and executed in INT8 on platforms supporting AVX-512
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you, I will try.
Is FP16 supported on CPU?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Andrey Malyshev (Intel) wrote:You need to follow "Low-Precision 8-bit Integer Inference" topic in in-package documentation.
Shortly, you have to use calibration tool which accept FP32 IR, calibration data set, will calibrate model and produce calibrated IR which will be able to be loaded to CPU plugin and executed in INT8 on platforms supporting AVX-512
Any guidelines on the size of the calibration data set? What are typical sizes used? 100 , 1000 , 10K ?
Never mind.. just found this - attaching for future reference.
..
The calibration dataset contains images and can be a subset of the validation set. A small fraction of images from validation dataset (1-5%) is enough to create a calibration dataset.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Stefano M. wrote:Thank you, I will try.
Is FP16 supported on CPU?
> Is FP16 supported on CPU?
I believe not supported on CPU according to the documentation. OpenVino team can confirm.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
ok thanks
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page