OpenZoo Model Runs on CPU not VPU (neural stick)

DaveInPA · ‎10-28-2021

Finally got my OpenVino environment up and running on my windows machine using Python. I started working with the "semantic-segmentation-adas-0001" demo model from the OpenZoo pretrained models. This model does the same as my initial model (ENet segmentation) that is uses a Lua/Torch framework. I tried all kinds of ways to get that model into OpenVino but i couldn't get it exported.

The adas-0001 runs great on my PC using the CPU without any errors or warnings. However when i run it using the Neural Stick 2 by adding 1 line of code i get errors. Here is my code to load the model...looks like its running out of memory??

I was able to download another zoo model (road-segmentation) and that one runs fine on the VPU and the CPU. So i know at least my environment and hardware is working properly.

net = cv2.dnn.readNet(model_bin, model_xml)
net.setPreferableTarget(cv2.dnn.DNN_TARGET_MYRIAD)

Here are the errors when i use the dnn.forward() method:

E: [global] [         0] [] XLinkOpenStream:73  Got wrong package from device, error code = X_LINK_OUT_OF_MEMORY
E: [ncAPI] [         0] [] ncFifoAllocate:3076  can't open stream:  FIFO0 due to not enough memory on device
F: [global] [         0] [EventRead00Thr] dispatcherEventReceive:91     Duplicate id detected. 

E: [global] [         0] [EventRead00Thr] handleIncomingEvent:543       Assertion Failed: event->header.type >= XLINK_WRITE_REQ && event->header.type != XLINK_REQUEST_LAST && event->header.type < XLINK_RESP_LAST

F: [global] [         0] [EventRead00Thr] dispatcherEventReceive:91     Duplicate id detected. 

E: [xLinkWinUsb] [         0] [Scheduler00Thr] usb_bulk_write:467
WinUsb_WritePipe failed with error:=31

E: [global] [         0] [Scheduler00Thr] dispatcherEventSend:53        Write failed (header) (err -2) | event XLINK_WRITE_REQ

E: [xLink] [         0] [Scheduler00Thr] sendEvents:998 Event sending failed
F: [xLink] [         0] [Scheduler00Thr] dispatcherResponseServe:740    no request for this response: XLINK_WRITE_RESP 1

E: [global] [         0] [EventRead00Thr] handleIncomingEvent:563       handleIncomingEvent() Read failed -2

E: [xLinkWinUsb] [         0] [Scheduler00Thr] usb_bulk_write:467
WinUsb_WritePipe failed with error:=22

E: [global] [         0] [Scheduler00Thr] dispatcherEventSend:53        Write failed (header) (err -2) | event XLINK_WRITE_RESP

E: [xLink] [         0] [Scheduler00Thr] sendEvents:998 Event sending failed
E: [xLinkWinUsb] [         0] [Scheduler00Thr] usb_bulk_write:467
WinUsb_WritePipe failed with error:=22

E: [global] [         0] [Scheduler00Thr] dispatcherEventSend:53        Write failed (header) (err -2) | event XLINK_READ_REL_REQ

E: [xLink] [         0] [Scheduler00Thr] sendEvents:998 E: [ncAPI] [         0] [] checkGraphMonitorResponse:1930       Graph monitor request returned error 1936090624
Event sending failed
E: [xLinkWinUsb] [         0] [Scheduler00Thr] usb_bulk_write:467
WinUsb_WritePipe failed with error:=22

E: [global] [         0] [Scheduler00Thr] dispatcherEventSend:53        Write failed (header) (err -2) | event XLINK_WRITE_REQ

E: [xLink] [         0] [Scheduler00Thr] sendEvents:998 Event sending failed
E: [global] [         0] [] addEvent:264        Condition failed: event->header.flags.bitField.ack != 1
E: [global] [         0] [] addEventWithPerf:276         addEvent(event) method call failed with an error: 3
E: [global] [         0] [] XLinkReadData:156   Condition failed: (addEventWithPerf(&event, &opTime))
E: [ncAPI] [         0] [] getGraphMonitorResponseValue:1898    XLink error, rc: X_LINK_ERROR
Traceback (most recent call last):
  File "gp.py", line 182, in <module>
    output = net.forward()
cv2.error: OpenCV(4.5.3-openvino) C:\jenkins\workspace\OpenCV\OpenVINO\2021.4\build\windows\opencv\modules\dnn\src\ie_ngraph.cpp:738: error: (-2:Unspecified error) Failed to initialize Inference Engine backend (device = MYRIAD): Failed to create input FIFO: NC_OUT_OF_MEMORY in function 'cv::dnn::InfEngineNgraphNet::initPlugin'

Wan_Intel · ‎10-29-2021

Hi DaveInPA,

Thank you for reaching out to us and thank you for using Intel® NCS2!

From Intel’s Pre-Trained Models Device Support, the semantic-segmentation-adas-0001 model is supported by the MYRIAD plugin.

From your information, your application was able to work with the road-segmentation-adas-0001 model using Intel® NCS2, but encountered an error: NC_OUT_OF_MEMORY when working with the semantic-segmentation-adas-0001 model.

I noticed that the default input shape for semantic-segmentaion-adas-0001 model [1, 3, 1024, 2048] was larger than the default input shape for road-segmentation-adas-0001 model [1, 3, 512, 896].

There is one similar case from the OpenVINO community. You may refer to here for more information.

When we executed the mask_rcnn_resnet50_atrous_coco model (default input shape was [1, 800, 1365, 3] ) with Benchmark C++ Tool using Intel® NCS2, we encountered system runs out of memory.

However, when we change the input shape to [1, 3, 224, 224], the application ran successfully. Please try to reduce the input shape and re-run your application again.

If the problem still exists, could you please share which demo application are you using from your end?

Also, please share your scripts with us so we can further investigate this issue from our side.

Regards,

Wan

DaveInPA · ‎10-29-2021

Hi - Because its a pre-trained model i don't think i can change the number of inputs can I? As you can see in my code below i started by cutting the dimensions in 1/2 but got an error.

	# construct a blob from the frame and perform a forward pass
	# using the segmentation model
	frame = imutils.resize(frame, width=args["width"])
	print("[INFO] Creating blob...")
	#blob = cv2.dnn.blobFromImage(frame, 1 , (2048, 1024), 1, swapRB=True, crop=False)
	blob = cv2.dnn.blobFromImage(frame, 1 , (1024, 512), 1, swapRB=True, crop=False)

	print("[INFO] Moving forward...")
	net.setInput(blob)
	output = net.forward()

Wan_Intel · ‎11-02-2021

Hi DaveInPA,

Thanks for your patience.

I have modified segmentation.py based on your snippet of codes and ran the application with semantic-segmentation-adas-0001 and road-segmentation-adas-0001 using Intel® Neural Compute Stick 2.

I’m able to run the application successfully when using road-segmentation-adas-0001, but I encountered a similar error as you faced when using semantic-segmentation-adas-0001.

Also, I encountered a new error when I reduced the --height and --width as suggested in the previous post.

Based on the development team’s response, this is a known issue, and our developers are trying to fix it. Meanwhile, you may use the following networks for the OpenCV Semantic Segmentation application as these networks have been tested and known to work:

· FCN (Caffe)

· ENet (Torch)

· ResNet101_DUC_HDC (ONNX: https://github.com/onnx/models)

· DeepLab (TensorFlow)

· UNet, DeepLabV3, FPN from Segmentation Models PyTorch

· Unet, UNetPlus, BiSeNet from Human Segmenation PyTorch

List of tested models for OpenCV deep learning module samples are available at the following page:

https://github.com/opencv/opencv/wiki/Deep-Learning-in-OpenCV

On another note, semantic-segmentation-adas-0001 and road-segmentation-adas-0001 can be used with the following segmentation demo provided by Open Model Zoo:

· Image Segmentation Python Demo

· Image Segmentation C++ Demo

Hope it helps.

Regards,

Wan

DaveInPA · ‎11-02-2021

Thank you for following up on this!

The ENet model is actually what i started with but couldn't get that to run in OpenVino. I will open a new thread on that and see if anyone has successfully ran that model and maybe has run into the issue i am running into.

Regarding your note about the demos provided by Open Model Zoo - are you saying those apps will run on the Compute Stick with the road and semantic segmentation-adas models?

Wan_Intel · ‎11-02-2021

Hi DaveInPA,

Thanks for your information.

We have received your new question in the OpenVINO™ community at the following thread:

https://community.intel.com/t5/forums/forumtopicpage/board-id/distribution-openvino-toolkit/message-id/25669#M25669

We are now working on it and will get back to you at the OpenVINO™ community thread above soon.

On the other hand, Image Segmentation Python Demo and Image Segmentation C++ Demo can run on the Intel® Neural Compute Stick 2 with the road-segmentation-adas-0001 model and semantic-segmentation-adas-0001 model.

The inference results are as follow:

1. Image Segmentation Python Demo with road-segmentation-adas-0001 model using Intel® Neural Compute Stick 2:

2. Image Segmentation C++ Demo with road-segmentation-adas-0001 model using Intel® Neural Compute Stick 2:

3. Image Segmentation Python Demo with semantic-segmentation-adas-0001 model using Intel® Neural Compute Stick 2:

4. Image Segmentation C++ Demo with semantic-segmentation-adas-0001 model using Intel® Neural Compute Stick 2:

Best regards,

Wan

Wan_Intel · ‎11-14-2021

Hi DaveInPA,

Thank you for your question.

If you need any additional information from Intel, please submit a new question as this thread is no longer being monitored.

Regards,

Wan