Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6152 Discussions

Low-Precision 8-bit Integer Inference on NCS2

Dal_Ben__Mattia
Beginner
1,368 Views

Hi,

I haven't seen mentioned anywhere except here. Is it possibile? What are the steps required?

Thanks.

0 Kudos
1 Solution
Shubha_R_Intel
Employee
1,368 Views

Dear Dal Ben, Mattia,

I finally received an update.  

There is still no support for int8 in OpenVINO for VPU. And currently there is no clear plan for this feature as far as future releases go.

Thanks for using OpenVino !

Shubha

View solution in original post

12 Replies
Shubha_R_Intel
Employee
1,368 Views

Dear Dal Ben, Mattia,

I think you should find the following documentation quite helpful. I read through it and I don't see any exceptions for ncs2.

https://docs.openvinotoolkit.org/latest/_inference_engine_tools_calibration_tool_README.html

There aren't any MYRIAD related calibration tool exceptions noted in the 2019 R1 Release Notes though there are some exceptions related to calibration tool. Please do a search on "calibration" on the Release Notes page.

Hope it helps you and thanks for using OpenVino !

Shubha

 

 

nikos1
Valued Contributor I
1,368 Views

I believe low-Precision 8-bit Integer Inference is only for CPU / MKLDNN plug-in. No GPU or NCS(2) but I may be wrong. Let's try on NCS2 to find out for sure.

cheers

nikos

 

Shubha_R_Intel
Employee
1,368 Views

Oops. Nikos is absolutely right. There is no INT8 support on MYRIAD. I'm terribly sorry for the misinformation but I will file a bug on the documentation, because This Should Be Clear !

Thanks,

Shubha

Dal_Ben__Mattia
Beginner
1,368 Views

Ok thanks for the help! @nikos @Shubha

Dal_Ben__Mattia
Beginner
1,368 Views

Hi,

I'm writing here again because I'm receiving some mixed signal about the Myriad X INT8 support.

Here it's explicitly said that Myriad X has "Native FP16 and fixed point 8 bit support".

Is it a software support problem? Is this something will be available in the future?

Shubha_R_Intel
Employee
1,368 Views

Dear Dal Ben, Mattia,

Wow. This is definitely confusing ! I have inquired about this doc you mention within Intel. I will report my findings here.

Thanks,

Shubha

Dal_Ben__Mattia
Beginner
1,368 Views

@Shubha That would be really helpful, thank you. I'll wait for your reply.

Dal_Ben__Mattia
Beginner
1,368 Views

@Shubha Hi, any update on this?

Shubha_R_Intel
Employee
1,369 Views

Dear Dal Ben, Mattia,

I finally received an update.  

There is still no support for int8 in OpenVINO for VPU. And currently there is no clear plan for this feature as far as future releases go.

Thanks for using OpenVino !

Shubha

Jeff_at_UIUC
Novice
1,277 Views

Hi @Shubha_R_Intel, I just confirm to be sure if INT8 is still not being supported on Myriad Neural Compute Stick 2. Thanks!

Dal_Ben__Mattia
Beginner
1,368 Views

@Shubha R.

Thank you for the information.

Shubha_R_Intel
Employee
1,368 Views

Dal Ben, Mattia,

Of course. Thanks for using OpenVino !

Shubha

Reply