Community
cancel
Showing results for 
Search instead for 
Did you mean: 
liang__heng
Beginner
74 Views

is only device CPU suppports the 8-bit inference?

According to the "Low-Precision 8-bit Integer Inference",  only device cpu support the 8-bit inference , but I want to know why calibration tool  has the  command line argument "-d <device>"?

 

0 Kudos
1 Reply
Shubha_R_Intel
Employee
74 Views

Dear liang, heng

Right now OpenVino's INT8 calibration is designed for optimal performance on CPUs. You can try and optimize for other devices (like GPU or MYRIAD) but the performance will be bad. But really, there is nothing inherently stopping the calibration tools from working on devices outside of CPU.

I think the reason for -d is for compatibility with all our other tools, demos and samples which use -d. Also, for future portability there may be a way to perform calibration of models for other devices (obviously with good performance ) besides just CPU.

Hope it helps,

Shubha

Reply