Intel® Distribution of OpenVINO™ Toolkit
Community support and discussions about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all things computer vision-related on Intel® platforms.

openvino lower precision

rongrong__wang
Beginner
153 Views

https://docs.openvinotoolkit.org/latest/_docs_IE_DG_Int8Inference.html

in this document, i see the sentence:

This means that 8-bit inference can only be performed with the CPU plugin on the layers listed above. All other layers are executed in the format supported by the CPU plugin: 32-bit floating point format (fp32).

so, i should use the data format fp32 if i use the GPU and other layers that are not in the list?

Thank you!

0 Kudos
2 Replies
Shubha_R_Intel
Employee
153 Views

Dear rongrong, wang

INT8 inference is supported by GPU but the performance is not good on our current GPUs so I would advise against using it. 

Thanks,

Shubha

 

rongrong__wang
Beginner
153 Views

Shubha R. (Intel) wrote:

Dear rongrong, wang

INT8 inference is supported by GPU but the performance is not good on our current GPUs so I would advise against using it. 

Thanks,

Shubha

 

Thank you very much! I tried it and it really is like this

Reply