topic Re:Inference speed with INT8 model on a bit old CPU in Intel® Distribution of OpenVINO™ Toolkit

Inference speed with INT8 model on a bit old CPU

timosy — Fri, 22 Jul 2022 04:45:57 GMT

I prepared INT8 model, and tested it on a new CPU. I found that the inferece speed gets fast, this is OK. But, I tested the inference using the same model on a bit old CPU, I found that it does not get fast. How should I check whether CPU supports INT8 model or not?

Re:Inference speed with INT8 model on a bit old CPU

IntelSupport — Mon, 25 Jul 2022 09:37:26 GMT

Hi Timosy,

Thanks for reaching out.

The slow inference on old CPU is expected based on the hardware configuration and model layer framework. Refer to the Intel® Distribution of OpenVINO™ toolkit Benchmark Results for the inference performance on a specified hardware configuration.

Apart from that, INT8 is supported with CPU plugin. Check out the Supported Devices, Supported Model Formats, and Supported Layers documentation for more details.

Regards,

Aznie

Re:Inference speed with INT8 model on a bit old CPU

IntelSupport — Wed, 10 Aug 2022 05:16:16 GMT

Hi Timosy,

This thread will no longer be monitored since we have provided information. If you need any additional information from Intel, please submit a new question.

Regards,

Aznie