Intel® Compute Stick
Support for Intel® Compute Sticks and Cards
529 Discussions

Difference Output Results when run model on CPU and VPU

LAndr23
Beginner
522 Views

I have a model convert to IR (data type : FP16) and start inference.

But when compare the outputs when changing the device:

plugin = IEPlugin(device="CPU")

plugin = IEPlugin(device="GPU")

plugin = IEPlugin(device="MYRIAD")

 

The output I get from "CPU" or "GPU" are correct (or very little difference) but when run on "VPU", I get results are matrix with values "0".

 

What's happen with the VPU? Can you give me some reason?

 

Thanks.

 

 

0 Kudos
8 Replies
AlHill
Super User
395 Views

What is the model number of your compute stick?

 

Doc

LAndr23
Beginner
395 Views

I'm using the Intell NCS2

David_C_Intel
Employee
395 Views

Hi LAndr23,

 

Thanks for reaching out. Could you please answer the following:

 

  • On which OS are you using the Intel®  NCS2? 
  • Which OpenVINO™ toolkit version are you using?
  • Which sample / demo are you using?
  • Could you share the IR files and the model optimizer command you used?

 

Best regards,

David C.

LAndr23
Beginner
395 Views

Hi David C,

I'm working on Windows 10 64bit, OpenVINO version lastest 2020.2.

 

This's outputs I got when running on VPU device :

[['import/primaryQN/LSTM_hidden_cell_output': array([[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0.]], dtype=float32), 'import/primaryQN/LSTM_hidden_state_output': array([[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0.]], dtype=float32)}

 

But when running on CPU, outputs are not 0.

 

 

You can follow the attached files to get IR, PB file and my code I using to run on VPU.

 

Regards,

Andrew

David_C_Intel
Employee
395 Views

Hi LAndr23,

 

Thanks for the files provided.

Could you please send the command you used to convert the frozen model to IR format?

 

Regards,

David C.

LAndr23
Beginner
395 Views

Hi David C,

Thank for your reply.

This is my command convert the frozen model to IR.

python mo_tf.py -m D3RQN_phase3.pb --data_type FP16 --input "import/primaryQN/preContextsFlattened[1 1 248],import/primaryQN/LSTMCellZeroState/zeros[1 248],import/primaryQN/LSTMCellZeroState/zeros_1[1 248]" --output import/primaryQN/norm/Contexts,import/primaryQN/LSTM_hidden_cell_output,import/primaryQN/LSTM_hidden_state_output

David_C_Intel
Employee
395 Views

Hi LAndr23,

 

Thank you for the information.

We are currently looking into your issue, we will come back to you as soon as possible.

 

Regards,

David C.

David_C_Intel
Employee
395 Views

Hi LAndr23,

 

Thank you for your patience. It appears that it is not working as expected on the MYRIAD device because there are layers that are not supported by the VPU.

 

Before running inference on a model it is a good idea to always check for unsupported layers. Here is the code that will help you on this:

plugin1 = IECore() dev = "MYRIAD" #hetero_dev = "HETERO:MYRIAD,CPU" #dev = "CPU" #Read Model IR net = IENetwork(model=model_xml, weights=model_bin) # Get the supported layers of the network supported_layers = plugin1.query_network(network=net, device_name=dev) # Check for any unsupported layers, and let the user # know if anything is missing. Exit the program, if so. unsupported_layers = [l for l in net.layers.keys() if l not in supported_layers] if len(unsupported_layers) != 0: print("The following layers are not supported by the plugin for specified device {}:\n {}".format(dev, ', '.join(unsupported_layers))) print("Please try to specify extensions library path in sample's command line parameters using -l or device specific extension, i.e. --cpu-extension command line argument") sys.exit(1) input_blob = next(iter(net.inputs)) t = time.time() # Load model exec_net1 = plugin1.load_network(net, hetero_dev)

 

The output for your model is:

 

The following layers are not supported by the plugin for specified device MYRIAD:

 import/primaryQN/preContextsFlattened, import/primaryQN/LSTMCellZeroState/zeros_1, import/primaryQN/LSTMCellZeroState/zeros, import/primaryQN/Contexts/Cast_1588_const, Copy_import/primaryQN/mask/Output_0/Data__const, Copy_import/primaryQN/mask/Output_0/Data_267_const

 

 

If you need to use MYRIAD plugin then we recommend to use the CPU as a backup device that can handle the unsupported layers. That can be done with the Heterogeneous plugin.

 

Best regards,

David C.

Reply