in this document, i see the sentence:
This means that 8-bit inference can only be performed with the CPU plugin on the layers listed above. All other layers are executed in the format supported by the CPU plugin: 32-bit floating point format (fp32).
so, i should use the data format fp32 if i use the GPU and other layers that are not in the list?
Shubha R. (Intel) wrote:
Dear rongrong, wang
INT8 inference is supported by GPU but the performance is not good on our current GPUs so I would advise against using it.
Thank you very much! I tried it and it really is like this