NCS2 Deployment

[Morsali__Reza]

Hi,

I have my deep learning model running well on CPU and GPU via OpenVINO. When I specify the Neural Compute Stick 2 as my inference engine, however, I get incorrect prediction from NCS 2. I am using Anaconda command prompt on Windows 10. Could you please help me to sort out this issue? Thank you.

To run the code please unzip the attached file (all the necessary files e.g. input file and IR file are attached). 

For inferencing on  CPU:

python CNN.py -i X_one -m frozen_graph.xml -d cpu

For inferencing on  NCS 2:

python CNN.py -i X_one -m frozen_graph.xml -d MYRIAD

[Sahira_Intel]

Hi Reza,

We are still looking into this issue unfortunately we don't have anything to share at the moment. We don't have an ETA but will get back to you as soon as we have an update, thank you for your patience.

Best Regards,

Sahira

[SerkanUygungelen]

Hi Reza,

Even you use a model with FP16 precision, CPU uses FP32 whereas MYRIAD uses FP16 as a backend. Thus, it is normal to see such differences in your model. I have tried all your examples (with 2020.3), the delta between CPU and MYRIAD probabilities is in the range of 10^-5 (0.00001) which is fine and should not impact your output decision. In case you still see a different decision, could you specify the exact image name?

[Morsali__Reza]

Hi Serkan,

Thank you for your response. Yes, I can specify the exact image name in inference. The problem is that when I use MYRIAD for inferencing "classid 1" always has a higher probability. 

For instance, let's consider the four different sample inputs that are attached here. As the name of the files show two of the input files belong to class id 0 and the other two files belong to class id 1.

For inferencing:

python CNN.py -i input_Image -m frozen_graph.xml -d device

the input_image  in above command can be (name of the input files in the attached folder): X_one_0_1.npy, X_one2_1_0.npy, X_one3_1_0.npy, or X_one4_0_1.npy

the device can be: cpu or MYRIAD

CPU correctly predicts the right class for each input while when I use MYRIAD, "classid 1" always has a higher probability. 

Regards,

Reza

[Morsali__Reza]

Hi Serkan,

Please let me know if you need any information from me.

Regards,

Reza

[SerkanUygungelen]

Hi Reza,

The MYRIAD plugin uses FP16 precision and the smallest positive normal number it can support is 0.000061035. The input you are providing to the model has values smaller than the minimum supported by FP16 so they all become 0 in the plugin. As the input values lose their precisions, you get an output with the same probability for all input you have provided. To fix this problem, you need to scale the input values to fit FP16 range. Also, you also need to scale the weights for the first convolution.

Regards,

Serkan