Yolo v3 conversion to IR issues

[Jacob_K_Intel]

hey i have a university team that i manage for intel and they sent me a question that i don't know the answer to. also can't find the link to the internal forum. here is what the team sent.

This time around I think I have narrowed down our issue to the IR conversion step. 

1. We are using the YOLO architecture and trained weights have been tested with darknet. To convert weights to IR I have been following this link: https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_YOLO_From_Tensorflow.html ;
However, I get conversion data format errors (shape cannot be -1). The default yolov3.weights and coco models (referenced in the tutorial above) suffer from the same issue. 
I tried to follow notes from various intel websites/forums and was successful to generate a model by setting --batch=1. *Is this the right way to mitigate such an issue?* 

2. Further, after editing the yolo_v3.json file in /deployment_tools/model_optimizer/extensions/front/tf (to meet our cfg formats), the sample inference engine code exits with errors. After a lot of debugging, seems like the read model (xml) has only 3 outputs. This is true for our model (with 54 output classes) and coco model (80 output classes). *Does this sound like an issue with conversion to IR or something else?* 
Please advise 

3. Could you also provide some insight into the yolo_v3.json file and how to modify it. The documentation on the link above seems insufficient. 
 

[Shubha_R_Intel]

Dear  Jacob K.,

My answers are below:

1) add --batch 1 to the mo_tf.py command line. You can also do --input_shape [1,416,416,3] where the first position is batch size. 416x416 is HxW and 3 is the number of channels. Yes these are fine ways to mitigate the issue.

2) This should not happen. I have converted mo_tf.py yolo v3 a dozen times by now on OpenVino 2019 R1. One thing that the documentation doesn't mention is that you need to upgrade to Tensorflow 1.11. 1.12 would be ok too but don't go as high as 1.13 because Model Optimizer does not support Tensorflow 1.13. Also make sure you're using OpenVino 2019 R1 (the latest release)

3) The yolo_v3.json file should be modified exactly as the document you referred to above suggests. It says :

Replace the default values in custom_attributes with the parameters that follow the [yolo] title in the configuration file.

The issue is that the darknet website link is a dead link (I will file a bug on this). The correct link is this one below:

https://github.com/pjreddie/darknet/tree/master/cfg

Hope it helps. Thanks,

Shubha

[Kurup__Akhil]

Hi Shubha,

1. Thanks 

2. I will check the tensorflow version I am using. My whole setup is in ROS and am using the ros_openvino_toolkit. I don't think this supports 2019_R1 yet.

update: we are using tensorflow v1.11

3. For the yolo_v3.json file, could you tell me which yolo layer should we refer to? The architecture has 3 yolo layers and based on the one we choose, the mask field will change. The `classes`, `coords` and `num` fields remain the same but I am not sure about the `mask` and the `entry_points`.

Akhil Kurup,

Michigan Tech. Univ.,

Dept. of Electrical and Computer Engineering

[Shubha_R_Intel]

Dear Kurup, Akhil,

The link below explains mask (lord pjreddie answers the question himself ! )

https://github.com/pjreddie/darknet/issues/558

But if you look at each of the [yolo] in https://raw.githubusercontent.com/pjreddie/darknet/master/cfg/yolov3.cfg you will see something like this: 

[yolo]
mask = 3,4,5
anchors = 10,13,  16,30,  33,23,  30,61,  62,45,  59,119,  116,90,  156,198,  373,326
classes=80
num=9
jitter=.3
ignore_thresh = .7
truth_thresh = 1
random=1

So organize it that way also in your yolo_v3.json (if you even need mask at all) as above. But you only need one of these :

 "entry_points": ["detector/yolo-v3/Reshape", "detector/yolo-v3/Reshape_4", "detector/yolo-v3/Reshape_8"]

The entry_points is something Model Optimizer needs. Those are actual layers in a yolo V3 model.

Lastly, remember, the yolo_v3.json which MO needs is in json format. The default yolov3.cfg is not in json. So you will have to insert all those annoying colons, commas and braces. If you mess up the json format, you will most certainly suffer a model optimizer (MO) error.

Hope it helps. And thanks for using OpenVino !

Shubha

[Kurup__Akhil]

Dear Shubha,

Thanks for the insight!
Upon viewing my pbtxt file, there are several Reshape layers (1..11). Why do we choose only 3 out of these and how do we make a decision on which to choose?

Also, If I convert as is and run the model with the example provided in the ros_openvino_toolkit package, it exits with `[ERROR] object detection network should have only one output`. The default Yolo-v3 model has 3 output layers. Any idea on how do we circumvent this issue?

Thanks

Akhil Kurup