Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.

Low-Precision 8-bit Integer Inference on NCS2

Dal_Ben__Mattia
Beginner
1,864 Views

Hi,

I haven't seen mentioned anywhere except here. Is it possibile? What are the steps required?

Thanks.

0 Kudos
1 Solution
Shubha_R_Intel
Employee
1,864 Views

Dear Dal Ben, Mattia,

I finally received an update.  

There is still no support for int8 in OpenVINO for VPU. And currently there is no clear plan for this feature as far as future releases go.

Thanks for using OpenVino !

Shubha

View solution in original post

0 Kudos
12 Replies
Shubha_R_Intel
Employee
1,864 Views

Dear Dal Ben, Mattia,

I think you should find the following documentation quite helpful. I read through it and I don't see any exceptions for ncs2.

https://docs.openvinotoolkit.org/latest/_inference_engine_tools_calibration_tool_README.html

There aren't any MYRIAD related calibration tool exceptions noted in the 2019 R1 Release Notes though there are some exceptions related to calibration tool. Please do a search on "calibration" on the Release Notes page.

Hope it helps you and thanks for using OpenVino !

Shubha

 

 

0 Kudos
nikos1
Valued Contributor I
1,864 Views

I believe low-Precision 8-bit Integer Inference is only for CPU / MKLDNN plug-in. No GPU or NCS(2) but I may be wrong. Let's try on NCS2 to find out for sure.

cheers

nikos

 

0 Kudos
Shubha_R_Intel
Employee
1,864 Views

Oops. Nikos is absolutely right. There is no INT8 support on MYRIAD. I'm terribly sorry for the misinformation but I will file a bug on the documentation, because This Should Be Clear !

Thanks,

Shubha

0 Kudos
Dal_Ben__Mattia
Beginner
1,864 Views

Ok thanks for the help! @nikos @Shubha

0 Kudos
Dal_Ben__Mattia
Beginner
1,864 Views

Hi,

I'm writing here again because I'm receiving some mixed signal about the Myriad X INT8 support.

Here it's explicitly said that Myriad X has "Native FP16 and fixed point 8 bit support".

Is it a software support problem? Is this something will be available in the future?

0 Kudos
Shubha_R_Intel
Employee
1,864 Views

Dear Dal Ben, Mattia,

Wow. This is definitely confusing ! I have inquired about this doc you mention within Intel. I will report my findings here.

Thanks,

Shubha

0 Kudos
Dal_Ben__Mattia
Beginner
1,864 Views

@Shubha That would be really helpful, thank you. I'll wait for your reply.

0 Kudos
Dal_Ben__Mattia
Beginner
1,864 Views

@Shubha Hi, any update on this?

0 Kudos
Shubha_R_Intel
Employee
1,865 Views

Dear Dal Ben, Mattia,

I finally received an update.  

There is still no support for int8 in OpenVINO for VPU. And currently there is no clear plan for this feature as far as future releases go.

Thanks for using OpenVino !

Shubha

0 Kudos
Jeff_at_UIUC
Novice
1,773 Views

Hi @Shubha_R_Intel, I just confirm to be sure if INT8 is still not being supported on Myriad Neural Compute Stick 2. Thanks!

0 Kudos
Dal_Ben__Mattia
Beginner
1,864 Views

@Shubha R.

Thank you for the information.

0 Kudos
Shubha_R_Intel
Employee
1,864 Views

Dal Ben, Mattia,

Of course. Thanks for using OpenVino !

Shubha

0 Kudos
Reply