Error when Converting YOLOv3 to IR

[ateş__taylan]

Hi, 

I have to convert my custom class yolov3 weigths to IR. In order to do this, at first I converted the last.weights to last.pb by using this repo:  https://github.com/mystic123/tensorflow-yolo-v3

and this command:

python3 convert_weights_pb.py  --weights_file /home/taylan/yolov3_birdeye/converted_weights/darknet/last.weights --class_names /home/taylan/yolov3_birdeye/converted_weights/darknet/coco.names --size 416 --data_format NHWC

It succesfully converted the darknet weights to .pb file.

After that, in order to edit the yolo_v3.json file, I converted the .pb file to .txt and search for detector/yolo-v3/Reshape_ to edit "enrty_points"

and I found 

prefix/detector/yolo-v3/Reshape/shape

prefix/detector/yolo-v3/Reshape_1/shape

prefix/detector/yolo-v3/Reshape_2/shape

prefix/detector/yolo-v3/Reshape_3/shape

prefix/detector/yolo-v3/Reshape_4/shape

prefix/detector/yolo-v3/Reshape_5/shape

prefix/detector/yolo-v3/Reshape_6/shape

prefix/detector/yolo-v3/Reshape_7/shape

prefix/detector/yolo-v3/Reshape_8/shape

prefix/detector/yolo-v3/Reshape_9/shape

prefix/detector/yolo-v3/Reshape_10/shape

prefix/detector/yolo-v3/Reshape_11/shape

and my yolov3.json file is as follows:

[
  {
    "id": "TFYOLOV3",
    "match_kind": "general",
    "custom_attributes": {
      "classes": 1,
      "coords": 4,
      "num": 9,
      "mask": [0,1,2],
      "anchors":[10,13,16,30,33,23,30,61,62,45,59,119,116,90,156,198,373,326],
      "entry_points": ["detector/yolo-v3/Reshape", "detector/yolo-v3/Reshape_1", "detector/yolo-v3/Reshape_2", "detector/yolo-v3/Reshape_3", "detector/yolo-v3/Reshape_4", "detector/yolo-v3/Reshape_5", "detector/yolo-v3/Reshape_6", "detector/yolo-v3/Reshape_7", "detector/yolo-v3/Reshape_8", "detector/yolo-v3/Reshape_9", "detector/yolo-v3/Reshape_10", "detector/yolo-v3/Reshape_11"]
    }
  }
]

here is my yolov3.cfg file:

[yolo]
mask = 0,1,2
anchors = 10,13,  16,30,  33,23,  30,61,  62,45,  59,119,  116,90,  156,198,  373,326
classes=1
num=9
jitter=.3
ignore_thresh = .7
truth_thresh = 1
random=1

to convert .pb file to IR, I used the following command:

python3 /opt/intel/openvino_2019.3.376/deployment_tools/model_optimizer/mo_tf.py --input_model /home/taylan/tensorflow-yolo-v3/frozen_darknet_yolov3_model.pb --tensorflow_use_custom_operations_config /opt/intel/openvino_2019.3.376/deployment_tools/model_optimizer/extensions/front/tf/yolo_v3.json  --input_shape=[16,416,416,3] --data_type=FP32

And here is my errors:

Model Optimizer arguments:
Common parameters:
    - Path to the Input Model:     /home/taylan/yolov3_birdeye/converted_weights/tensorflow/last.pb
    - Path for generated IR:     /home/taylan/yolov3_birdeye/.
    - IR output name:     last
    - Log level:     ERROR
    - Batch:     Not specified, inherited from the model
    - Input layers:     Not specified, inherited from the model
    - Output layers:     Not specified, inherited from the model
    - Input shapes:     [16,416,416,3]
    - Mean values:     Not specified
    - Scale values:     Not specified
    - Scale factor:     Not specified
    - Precision of IR:     FP32
    - Enable fusing:     True
    - Enable grouped convolutions fusing:     True
    - Move mean values to preprocess section:     False
    - Reverse input channels:     False
TensorFlow specific parameters:
    - Input model in text protobuf format:     False
    - Path to model dump for TensorBoard:     None
    - List of shared libraries with TensorFlow custom layers implementation:     None
    - Update the configuration file with input/output node names:     None
    - Use configuration file used to generate the model with Object Detection API:     None
    - Operations to offload:     None
    - Patterns to offload:     None
    - Use the config file:     /opt/intel/openvino_2019.3.376/deployment_tools/model_optimizer/extensions/front/tf/yolo_v3.json
Model Optimizer version:     2019.3.0-408-gac8584cb7



[ ERROR ]  Cannot infer shapes or values for node "detector/yolo-v3/meshgrid/mul_1/YoloRegion".
[ ERROR ]  index 2 is out of bounds for axis 0 with size 2
[ ERROR ]  
[ ERROR ]  It can happen due to bug in custom shape infer function <function RegionYoloOp.regionyolo_infer at 0x7f86d0635268>.
[ ERROR ]  Or because the node inputs have incorrect values/shapes.
[ ERROR ]  Or because input shapes are incorrect (embedded to the model or passed via --input_shape).
[ ERROR ]  Run Model Optimizer with --log_level=DEBUG for more information.
[ ERROR ]  index 2 is out of bounds for axis 0 with size 2
Stopped shape/value propagation at "detector/yolo-v3/meshgrid/mul_1/YoloRegion" node. 
 For more information please refer to Model Optimizer FAQ (https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html), question #38. 
Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.PartialInfer.PartialInfer'>): index 2 is out of bounds for axis 0 with size 2
Stopped shape/value propagation at "detector/yolo-v3/meshgrid/mul_1/YoloRegion" node. 
 For more information please refer to Model Optimizer FAQ (https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html), question #38. 

my batch size during training is 16, so I used --input_shape=[16,416,416,3], I tried also [1,416,416,3] and apart from these, I tried --batch 1 and --batch 16, nothing changed.

How can I do in order to convert my weights to IR?

If I keep this line not changed:  "entry_points": ["detector/yolo-v3/Reshape", "detector/yolo-v3/Reshape_4", "detector/yolo-v3/Reshape_8"]

 the weights successfully converted to IR , however the model cannot find any objects.

Thank you in advance

Taylan

[ateş__taylan]

There is one update, I converted darknet weights to IR by keeping "entry_points" as  ["detector/yolo-v3/Reshape", "detector/yolo-v3/Reshape_4", "detector/yolo-v3/Reshape_8"] after visualising the original yolov3.pb and my custom yolov3.pb. However, the model cannot find the instances. may be because there should be a preprocessing before inference.

Am I correctly convert the darknet weights to IR? If so, the mistake should be in the inference code.

[ateş__taylan]

I successfully converted the official yolov3.weigths to IR and it is working with the object_detection_demo_yolov3_async.py. However, although I followed the right and the same steps and observed the .pb graph and edited the yolo_v3.json file according to the graph and my yolov3.cfg file, my one-class custom yolov3 model finds nothing with with openvino, I mean it cannot detect the objects. 

I am waiting for your reply,

Thank you in advance,

Taylan

[JesusE_Intel]

Hi Taylan,

Thanks for reaching out, could you provide me the frozen model (.pb) for me to try and a sample image/video? You should use [1,416,416,3] as the input shape. Also, try adding the --reverse_input_channels to the model optimizer command as this is needed when a model will be used with our sample applications.

The following command looks correct, I removed the --size 416 as that is the default value.

python3 convert_weights_pb.py  --weights_file /home/taylan/yolov3_birdeye/converted_weights/darknet/last.weights --class_names /home/taylan/yolov3_birdeye/converted_weights/darknet/coco.names --data_format NHWC

Let me know if you would like to share the model and sample input privatly and I will start a private message. 

Regards,

Jesus

[ateş__taylan]

Thank you for your reply Sir, 

I want to share the model and sample videos with you.

Thank you in advance, 

Taylan

[JesusE_Intel]

Thank you for sending over the files, I have sent you a private message requesting additional information.

Regards,

Jesus

[JesusE_Intel]

Hi Taylan,

As you mentioned in your private message, the issue was solved by changing the custom model configuration file. Please feel free to reach out again if you have additional questions.

Regards,

Jesus 

[pang__pih]

I have same error, can you share the solution to this problem? How your custom_yolo_v3.cfg should look like, so it have the same structure as the coco pretrained model?

Thanks

Pih Lung

[ateş__taylan]

Hello pang, the problem that I faced results from changing the original yolov3.cfg file. I used the original yolov3.cfg file as 80 class and my dataset consists of only one class, and the problem is solved. But now, I am using yolov3-1cls.cfg in ultralytics yolov3 github repository and didn't try on OpenVINO toolkit, but it is working on OpenCV dnn module. Therefore, do not change the original cfg files. My problem results from changing the original cfg files.