OpenCV using Movidius NCS2 gives error when using Darknet neural network

[ggeol]

I am trying to perform OCR using the Intel Movidius Neural Compute Stick 2. The OCR network that I am using is based on YOLO, and therefore has the graph structure saved as a  `.cfg` file and the weights as a `.weights` file. I use OpenCV which was installed with Openvino on a Raspberry Pi 3b+. My code is given below:

import cv2   # load network ocr_net = cv2.dnn.readNetFromDarknet('ocr-net.cfg', 'ocr-net.weights') ocr_net.setPreferableTarget(cv2.dnn.DNN_TARGET_MYRIAD) ocr_net.setPreferableBackend(cv2.dnn.DNN_BACKEND_INFERENCE_ENGINE)   # perform inference im = cv2.imread('img_path.jpg') blob = cv2.dnn.blobFromImage(im, swapRB=False) ocr_net.setInput(blob) result = ocr_net.forward()

On running this code, I get the following error:

Traceback (most recent call last): File "test.py", line 12, in <module> result = ocr_net.forward() cv2.error: OpenCV(4.1.2-openvino) /home/jenkins/workspace/OpenCV/OpenVINO/build/opencv/modules/dnn/src/op_inf_engine.cpp:704: error: (-215:Assertion failed) Failed to initialize Inference Engine backend: Device with "CPU" name is not registered in the InferenceEngine in function 'initPlugin'

If I change the inference target to `cv2.dnn.DNN_TARGET_CPU` and the inference backend to `cv2.dnn.DNN_BACKEND_OPENCV`, the model works fine and is able to correctly perform OCR. My openvino/openCV setup is also correctly installed since I can correctly run other Tensorflow/Caffe models using openCV (using `readNetFromTensorflow()`/`readNetFromCaffe()`).

The OCR network that I am attempting to use can be found here: cfg , weights .

Any help is appreciated!

[MAURICIOAL_R_Intel]

Hello ggeol,

This issue may be specific to OpenCV and our recommendation for you is to open a new GitHub issue in the following link: https://github.com/opencv/opencv/issues.

In addition, you can try to read your model in IR format instead of darknet (.cfg & .weights) as shown in the setup guide: https://docs.openvinotoolkit.org/latest/_docs_install_guides_installing_openvino_raspbian.html#run-inference-opencv

To convert the model from darknet to IR format, please follow these instructions: https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_YOLO_From_Tensorflow.html

If you have more questions, please feel free to ask us, and we will help you further. 

Thank you. 

Mauricio R. 

Intel Customer Support Technician 

A Contingent Worker at Intel 

[ggeol]

Hi @Mauricio.R_Intel​ ,

Thanks for your reply. I have opened up a issue on OpenCV as suggested.

I have tried the steps given to convert the model to the IR graph. I am able to get the tensorflow graph, but when i try to convert the tensorflow graph to IR i get the following error message:

Model Optimizer arguments: Common parameters: - Path to the Input Model: path\to\ocr-net.pb - Path for generated IR: C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\model_optimizer\. - IR output name: ocr-net - Log level: ERROR - Batch: 1 - Input layers: Not specified, inherited from the model - Output layers: Not specified, inherited from the model - Input shapes: Not specified, inherited from the model - Mean values: Not specified - Scale values: Not specified - Scale factor: Not specified - Precision of IR: FP32 - Enable fusing: True - Enable grouped convolutions fusing: True - Move mean values to preprocess section: False - Reverse input channels: False TensorFlow specific parameters: - Input model in text protobuf format: False - Path to model dump for TensorBoard: None - List of shared libraries with TensorFlow custom layers implementation: None - Update the configuration file with input/output node names: None - Use configuration file used to generate the model with Object Detection API: None - Operations to offload: None - Patterns to offload: None - Use the config file: C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\model_optimizer\extensions\front\tf\yolo_v1_tiny.json Model Optimizer version: 2019.3.0-375-g332562022 OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids. OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 0-7 OMP: Info #156: KMP_AFFINITY: 8 available OS procs OMP: Info #157: KMP_AFFINITY: Uniform topology OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 2 threads/core (4 total cores) OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map: OMP: Info #171: KMP_AFFINITY: OS proc 0 maps to package 0 core 0 thread 0 OMP: Info #171: KMP_AFFINITY: OS proc 1 maps to package 0 core 0 thread 1 OMP: Info #171: KMP_AFFINITY: OS proc 2 maps to package 0 core 1 thread 0 OMP: Info #171: KMP_AFFINITY: OS proc 3 maps to package 0 core 1 thread 1 OMP: Info #171: KMP_AFFINITY: OS proc 4 maps to package 0 core 2 thread 0 OMP: Info #171: KMP_AFFINITY: OS proc 5 maps to package 0 core 2 thread 1 OMP: Info #171: KMP_AFFINITY: OS proc 6 maps to package 0 core 3 thread 0 OMP: Info #171: KMP_AFFINITY: OS proc 7 maps to package 0 core 3 thread 1 OMP: Info #250: KMP_AFFINITY: pid 12488 tid 2984 thread 0 bound to OS proc set 0 OMP: Info #250: KMP_AFFINITY: pid 12488 tid 2664 thread 1 bound to OS proc set 2 OMP: Info #250: KMP_AFFINITY: pid 12488 tid 16768 thread 2 bound to OS proc set 4 OMP: Info #250: KMP_AFFINITY: pid 12488 tid 5540 thread 3 bound to OS proc set 6 [ ERROR ] Cannot infer shapes or values for node "output/YoloRegion". [ ERROR ] 'mask' [ ERROR ] [ ERROR ] It can happen due to bug in custom shape infer function <function RegionYoloOp.regionyolo_infer at 0x000001E299EBAEA0>. [ ERROR ] Or because the node inputs have incorrect values/shapes. [ ERROR ] Or because input shapes are incorrect (embedded to the model or passed via --input_shape). [ ERROR ] Run Model Optimizer with --log_level=DEBUG for more information. [ ERROR ] 'mask' Stopped shape/value propagation at "output/YoloRegion" node. For more information please refer to Model Optimizer FAQ (https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html), question #38. Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.PartialInfer.PartialInfer'>): 'mask' Stopped shape/value propagation at "output/YoloRegion" node. For more information please refer to Model Optimizer FAQ (https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html), question #38.

I believe this error comes form the fact that this is not exactly the YOLO model. How to I convert a custom darknet model to the IR format? If it helps, when I directly try to use the `.pb` file with OpenCV, I get the following error:

File "test.py", line 12 ocr_net.forward() cv2.error: OpenCV(4.1.0) C:\projects\opencv-python\opencv\modules\dnn\src\dnn.cpp:524: error: (-2:Unspecified error) Can't create layer "init" of type "NoOp" in function 'cv::dnn::dnn4_v20190122::LayerData::getLayerInstance'

[MAURICIOAL_R_Intel · Accepted solution]

Hello ggeol,

In this case, we only have a guide for converting yolo models from DarkNet. Since your model is not the standard yolo model, it may not be supported, and that is why we recommended opening a new issue as you did. You can try to modify the .json file to match his model but it may or may not work.

Thank you.