Low-Precision 8-bit Integer Inference on NCS2

[Dal_Ben__Mattia]

Hi,

I haven't seen mentioned anywhere except here. Is it possibile? What are the steps required?

Thanks.

[Shubha_R_Intel]

Dear Dal Ben, Mattia,

I think you should find the following documentation quite helpful. I read through it and I don't see any exceptions for ncs2.

https://docs.openvinotoolkit.org/latest/_inference_engine_tools_calibration_tool_README.html

There aren't any MYRIAD related calibration tool exceptions noted in the 2019 R1 Release Notes though there are some exceptions related to calibration tool. Please do a search on "calibration" on the Release Notes page.

Hope it helps you and thanks for using OpenVino !

Shubha

[nikos1]

I believe low-Precision 8-bit Integer Inference is only for CPU / MKLDNN plug-in. No GPU or NCS(2) but I may be wrong. Let's try on NCS2 to find out for sure.

cheers

nikos

[Shubha_R_Intel]

Oops. Nikos is absolutely right. There is no INT8 support on MYRIAD. I'm terribly sorry for the misinformation but I will file a bug on the documentation, because This Should Be Clear !

Thanks,

Shubha

[Dal_Ben__Mattia]

Ok thanks for the help! @nikos @Shubha

[Dal_Ben__Mattia]

Hi,

I'm writing here again because I'm receiving some mixed signal about the Myriad X INT8 support.

Here it's explicitly said that Myriad X has "Native FP16 and fixed point 8 bit support".

Is it a software support problem? Is this something will be available in the future?

[Shubha_R_Intel]

Dear Dal Ben, Mattia,

Wow. This is definitely confusing ! I have inquired about this doc you mention within Intel. I will report my findings here.

Thanks,

Shubha

[Dal_Ben__Mattia]

@Shubha That would be really helpful, thank you. I'll wait for your reply.

[Dal_Ben__Mattia]

@Shubha Hi, any update on this?

[Shubha_R_Intel · Accepted solution]

Dear Dal Ben, Mattia,

I finally received an update.  

There is still no support for int8 in OpenVINO for VPU. And currently there is no clear plan for this feature as far as future releases go.

Thanks for using OpenVino !

Shubha

[Jeff_at_UIUC]

Hi @Shubha_R_Intel, I just confirm to be sure if INT8 is still not being supported on Myriad Neural Compute Stick 2. Thanks!

[Dal_Ben__Mattia]

@Shubha R.

Thank you for the information.

[Shubha_R_Intel]

Dal Ben, Mattia,

Of course. Thanks for using OpenVino !

Shubha