Community
cancel
Showing results for 
Search instead for 
Did you mean: 
LAndr23
Beginner
420 Views

Difference Output Results when run model on CPU and VPU

I have a model convert to IR (data type : FP16) and start inference.

But when compare the outputs when changing the device:

plugin = IEPlugin(device="CPU")

plugin = IEPlugin(device="GPU")

plugin = IEPlugin(device="MYRIAD")

 

The output I get from "CPU" or "GPU" are correct (or very little difference) but when run on "VPU", I get results are matrix with values "0".

 

What's happen with the VPU? Can you give me some reason?

 

Thanks.

 

 

0 Kudos
8 Replies
AlHill
Super User
293 Views

What is the model number of your compute stick?

 

Doc

LAndr23
Beginner
293 Views

I'm using the Intell NCS2

David_C_Intel
Employee
293 Views

Hi LAndr23,

 

Thanks for reaching out. Could you please answer the following:

 

  • On which OS are you using the Intel®  NCS2? 
  • Which OpenVINO™ toolkit version are you using?
  • Which sample / demo are you using?
  • Could you share the IR files and the model optimizer command you used?

 

Best regards,

David C.

LAndr23
Beginner
293 Views

Hi David C,

I'm working on Windows 10 64bit, OpenVINO version lastest 2020.2.

 

This's outputs I got when running on VPU device :

[['import/primaryQN/LSTM_hidden_cell_output': array([[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0.]], dtype=float32), 'import/primaryQN/LSTM_hidden_state_output': array([[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,

    0., 0., 0., 0., 0., 0., 0., 0.]], dtype=float32)}

 

But when running on CPU, outputs are not 0.

 

 

You can follow the attached files to get IR, PB file and my code I using to run on VPU.

 

Regards,

Andrew

David_C_Intel
Employee
293 Views

Hi LAndr23,

 

Thanks for the files provided.

Could you please send the command you used to convert the frozen model to IR format?

 

Regards,

David C.

LAndr23
Beginner
293 Views

Hi David C,

Thank for your reply.

This is my command convert the frozen model to IR.

python mo_tf.py -m D3RQN_phase3.pb --data_type FP16 --input "import/primaryQN/preContextsFlattened[1 1 248],import/primaryQN/LSTMCellZeroState/zeros[1 248],import/primaryQN/LSTMCellZeroState/zeros_1[1 248]" --output import/primaryQN/norm/Contexts,import/primaryQN/LSTM_hidden_cell_output,import/primaryQN/LSTM_hidden_state_output

David_C_Intel
Employee
293 Views

Hi LAndr23,

 

Thank you for the information.

We are currently looking into your issue, we will come back to you as soon as possible.

 

Regards,

David C.

David_C_Intel
Employee
293 Views

Hi LAndr23,

 

Thank you for your patience. It appears that it is not working as expected on the MYRIAD device because there are layers that are not supported by the VPU.

 

Before running inference on a model it is a good idea to always check for unsupported layers. Here is the code that will help you on this:

plugin1 = IECore() dev = "MYRIAD" #hetero_dev = "HETERO:MYRIAD,CPU" #dev = "CPU" #Read Model IR net = IENetwork(model=model_xml, weights=model_bin) # Get the supported layers of the network supported_layers = plugin1.query_network(network=net, device_name=dev) # Check for any unsupported layers, and let the user # know if anything is missing. Exit the program, if so. unsupported_layers = [l for l in net.layers.keys() if l not in supported_layers] if len(unsupported_layers) != 0: print("The following layers are not supported by the plugin for specified device {}:\n {}".format(dev, ', '.join(unsupported_layers))) print("Please try to specify extensions library path in sample's command line parameters using -l or device specific extension, i.e. --cpu-extension command line argument") sys.exit(1) input_blob = next(iter(net.inputs)) t = time.time() # Load model exec_net1 = plugin1.load_network(net, hetero_dev)

 

The output for your model is:

 

The following layers are not supported by the plugin for specified device MYRIAD:

 import/primaryQN/preContextsFlattened, import/primaryQN/LSTMCellZeroState/zeros_1, import/primaryQN/LSTMCellZeroState/zeros, import/primaryQN/Contexts/Cast_1588_const, Copy_import/primaryQN/mask/Output_0/Data__const, Copy_import/primaryQN/mask/Output_0/Data_267_const

 

 

If you need to use MYRIAD plugin then we recommend to use the CPU as a backup device that can handle the unsupported layers. That can be done with the Heterogeneous plugin.

 

Best regards,

David C.

Reply