Intel NCS2 Inference Error

[Tenescu__Andrei]

Hello, I am using the openvino_2019.2.242 toolkit and trying to run a GAN exported from Keras to ONNX and then converted to IR on the Intel Neural Compute Stick 2. I am trying to run it using the benchmark_app, in both it's compiled and Python versions. 

The model runs on the CPU, but fails when I try to run it on the NCS2 (using the -d MYRIAD option):

python benchmark_app.py -m /home/tzeny/Other/Projects/XVision/Benchmarks/BogdanKerasONNX/OpenVinoFP16/bogdan_onlyh5.xml -d MYRIAD
[Step 1/11] Parsing and validating input arguments
[Step 2/11] Loading Inference Engine
[ INFO ] InferenceEngine:
         2.0.custom_releases/2019/R2_f5827d4773ebbe727c9acac5f007f7d94dd4be4e
[ INFO ] Device is MYRIAD
         MYRIAD
         myriadPlugin............ version 2.0
         Build................... 27579

[Step 3/11] Read the Intermediate Representation of the network
[Step 4/11] Resizing network to match image sizes and given batch
[ INFO ] Network batch size: 1, precision MIXED
[Step 5/11] Configuring input of the model
[Step 6/11] Setting device configuration
[Step 7/11] Loading the model to the device
[Info   ][VPU][MyriadPlugin] Device #0 MYRIAD-X (USB protocol) allocated
E: [xLink] [    395328] [EventRead00Thr] handleIncomingEvent:267	handleIncomingEvent() Read failed (err -4)

E: [xLink] [    395328] [Scheduler00Thr] dispatcherEventSend:923	Write failed (event) (err -4)

E: [xLink] [    395328] [EventRead00Thr] dispatcherEventReceive:336	dispatcherEventReceive() Read failed (err -4) | event 0x7fc94e701e40 XLINK_WRITE_REQ

E: [xLink] [    395328] [EventRead00Thr] eventReader:223	eventReader thread stopped (err -4)
E: [xLink] [    395328] [python] XLinkReadDataWithTimeOut:1323	Event data is invalid
E: [ncAPI] [    395328] [python] ncGraphAllocate:1911	Can't read input tensor descriptors of the graph, rc: X_LINK_ERROR
E: [watchdog] [    395328] [WatchdogThread] sendPingMessage:120	Failed send ping message: X_LINK_ERROR
[ ERROR ] Failed to allocate graph: NC_ERROR
Traceback (most recent call last):
  File "/opt/intel/openvino_2019.2.242/deployment_tools/inference_engine/samples/python_samples/benchmark_app/benchmark/benchmark.py", line 193, in main
    num_requests=args.number_infer_requests if args.number_infer_requests else 0)
  File "ie_api.pyx", line 85, in openvino.inference_engine.ie_api.IECore.load_network
  File "ie_api.pyx", line 92, in openvino.inference_engine.ie_api.IECore.load_network
RuntimeError: Failed to allocate graph: NC_ERROR
W: [ncAPI] [    395342] [python] ncDeviceClose:1629	Failed to close stream, rc: X_LINK_ERROR
W: [ncAPI] [    395342] [python] ncDeviceClose:1635	Failed to close stream, rc: X_LINK_ERROR
W: [ncAPI] [    395342] [python] ncDeviceClose:1647	Failed to reset, rc: X_LINK_ERROR

I tried running another model (a basic densenet121) on the NCS2 and it worked. Do you have any idea what I'm doing wrong?

[Sahira_Intel]

Hi Andrei,

Can you please upgrade to the latest version of the OpenVINO Toolkit 2019 R3 and try again? 

Let me know if you have any further questions!

Sincerely,
Sahira 

[Tenescu__Andrei]

I have upgraded to openvino_2019.3.376, and I get this error:

./benchmark_app -m /opt/intel/openvino_2019.3.376/deployment_tools/model_optimizer/./bogdan_onlyh5.xml -d MYRIAD
[Step 1/11] Parsing and validating input arguments
[ INFO ] Parsing input parameters
[ WARNING ] -nstreams default value is determined automatically for a device. Although the automatic selection usually provides a reasonable performance,but it still may be non-optimal for some cases, for more information look at README.

[Step 2/11] Loading Inference Engine
[ INFO ] InferenceEngine: 
	API version ............ 2.1
	Build .................. custom_releases/2019/R3_ac8584cb714a697a12f1f30b7a3b78a5b9ac5e05
	Description ....... API
[ INFO ] Device info: 
	MYRIAD
	myriadPlugin version ......... 2.1
	Build ........... 32974

[Step 3/11] Reading the Intermediate Representation network
[ INFO ] Loading network files
[ INFO ] Read network took 36.00 ms
[Step 4/11] Resizing network to match image sizes and given batch
[ INFO ] Network batch size: 1, precision: MIXED
[Step 5/11] Configuring input of the model
[Step 6/11] Setting device configuration
[Step 7/11] Loading the model to the device
E: [global] [    484330] [Scheduler00Thr] dispatcherEventSend:1013	Write failed -4

E: [xLink] [    484331] [EventRead00Thr] eventReader:218	eventReader thread stopped (err -4)
E: [global] [    484331] [Scheduler00Thr] dispatcherEventSend:1004	Write failed (header) (err -4) | event XLINK_WRITE_RESP

E: [xLink] [    484331] [Scheduler00Thr] eventSchedulerRun:626	Event sending failed
E: [xLink] [    484331] [Scheduler00Thr] eventSchedulerRun:576	Dispatcher received NULL event!
E: [global] [    484331] [benchmark_app] XLinkReadDataWithTimeOut:1494	Event data is invalid
E: [ncAPI] [    484331] [benchmark_app] ncGraphAllocate:1947	Can't read output tensor descriptors of the graph, rc: X_LINK_ERROR
E: [watchdog] [    484331] [WatchdogThread] sendPingMessage:121	Failed send ping message: X_LINK_ERROR
W: [ncAPI] [    484340] [benchmark_app] ncDeviceClose:1662	Failed to close stream, rc: X_LINK_ERROR
W: [ncAPI] [    484340] [benchmark_app] ncDeviceClose:1666	Failed to close stream, rc: X_LINK_ERROR
W: [ncAPI] [    484340] [benchmark_app] ncDeviceClose:1678	Failed to reset, rc: X_LINK_ERROR
E: [ncAPI] [    494340] [benchmark_app] ncDeviceClose:1737	Device didn't appear after reboot
[ ERROR ] Failed to allocate graph: NC_ERROR

Running the model on the CPU (./benchmark_app -m /opt/intel/openvino_2019.3.376/deployment_tools/model_optimizer/./bogdan_onlyh5.xml -d CPU) works.

Running a DesneNet121 exported form PyTorch on the NCS2 works.

[Sahira_Intel]

Hi Andrei,

Can you please provide your model so I can run it on my end? If you'd like to send it over a PM, please let me know and I will message you. 

Sincerely,

Sahira 

[Tenescu__Andrei]

Hello,

Sadly the model I am working with is private Intelectual Property, belonging to the company I work for so I cannot send it to you. If you have any other suggestions I would be happy to try them :).

Thank you,

Andrei

[Sahira_Intel]

Hi Andrei,

I understand. Can you please try upgrading OpenVINO to the latest version and try again? 

Since your model runs on the CPU but not on the NCS2, there might be an issue when converting from Keras, or when converting from ONNX to IR. Can you please provide the command you used to convert your model to IR to run on the NCS2? 

Best Regards,

Sahira 

[Tenescu__Andrei]

Hello Sahira,

I am using the latest vesion (openvino_2019.3.376), this is the command I used:

python3 mo.py --input_model ~/ONNX/bogdan_onlyh5.onnx --input_shape "[1,1024,1024,1]" --data_type=FP16

Thank you for your help,

Andrei

[Sahira_Intel]

Hi Andrei,

Since this was a custom trained model, it could be possible that it is not supported on Myriad, which is probably why it runs on the CPU but not on the NCS2. Can you give me some more information about your model:

• What base model was used for training?
• What Framework & Topology is the model? 

Sincerely,
Sahira