Model Optimizer error with Yolo_v3 model

[SC_Huang]

I was converting a yolo_v3 model with following arguments

sudo python3 mo_tf.py --input_model ~/openvino_models/models/Customer/yolov3_cpu_nms.pb --tensorflow_use_custom_operations_config extensions/front/tf/yolo_v3.json --input_shape=[1,416,416,3] --input=Placeholder:0 --output=concat_9:0,mul_6:0 --data_type=FP32 --log_level=DEBUG

and get error like this

[ ERROR ]  Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.front.input_cut.InputCut'>): Placeholder node "Placeholder" doesn't have input port, but input port 0 was provided. 

How to slove this problem?

For more information, I've attach debug log and frozen tensorflow model file.

Thank you.

[Shubha_R_Intel]

Dear SC Huang,

you should not have to add --input=Placeholder:0 --output=concat_9:0,mul_6:0  . If you read the Model Optimizer Yolo V3 doc those switches are not needed.

Can you re-try without those switches ?

Thanks kindly,

Shubha

[SC_Huang]

Shubha R. (Intel) wrote:

Dear SC Huang,

you should not have to add --input=Placeholder:0 --output=concat_9:0,mul_6:0  . If you read the Model Optimizer Yolo V3 doc those switches are not needed.

Can you re-try without those switches ?

Thanks kindly,

Shubha

Hi Shubha,

I have tried, error occured as following

[ ERROR ]  Exception occurred during running replacer "TFYOLOV3" (<class 'extensions.front.tf.YOLO.YoloV3RegionAddon'>): TensorFlow YOLO V3 conversion mechanism was enabled. Entry points "detector/yolo-v3/Reshape, detector/yolo-v3/Reshape_4, detector/yolo-v3/Reshape_8" were provided in the configuration file. Entry points are nodes that feed YOLO Region layers. Node with name detector/yolo-v3/Reshape doesn't exist in the graph. Refer to documentation about converting YOLO models for more information.

This is the command i tried.

sudo python3 mo_tf.py --input_model ~/openvino_models/models/Customer/yolov3_cpu_nms.pb --tensorflow_use_custom_operations_config extensions/front/tf/yolo_v3.json --data_type=FP32

[Shubha_R_Intel]

Dear SC Huang,

OK, I believe you. Let me try it myself and see what happens. Maybe there is a bug.

Sorry for the trouble !

Shubha

[Shubha_R_Intel]

Dear SC Huang

 I just now tried yolo_v3 following exactly the steps outlined in Convert YOLO Model Optimizer Doc and had no issues. Below is the command I used: 

python  "c:\Program Files (x86)\IntelSWTools\openvino_2019.1.148\deployment_tools\model_optimizer\mo_tf.py" --input_model frozen_darknet_yolov3_model.pb --tensorflow_use_custom_operations_config "c:\Program Files (x86)\IntelSWTools\openvino_2019.1.148\deployment_tools\model_optimizer\extensions\front\tf\yolo_v3.json" --batch 1

And I used OpenVino 2019R1.1.

My guess is that you didn't install Model Optimizer pre-requisites.

Thanks,

Shubha

[SC_Huang]

Shubha R. (Intel) wrote:

Dear SC Huang

 I just now tried yolo_v3 following exactly the steps outlined in Convert YOLO Model Optimizer Doc and had no issues. Below is the command I used: 

python  "c:\Program Files (x86)\IntelSWTools\openvino_2019.1.148\deployment_tools\model_optimizer\mo_tf.py" --input_model frozen_darknet_yolov3_model.pb --tensorflow_use_custom_operations_config "c:\Program Files (x86)\IntelSWTools\openvino_2019.1.148\deployment_tools\model_optimizer\extensions\front\tf\yolo_v3.json" --batch 1

And I used OpenVino 2019R1.1.

My guess is that you didn't install Model Optimizer pre-requisites.

Thanks,

Shubha

If you were using model from the https://github.com/mystic123/tensorflow-yolo-v3 , I've succesfully done with that before by following steps from your websites.

But when it comes to my Yolo-v3 model as attachment "yolov3_cpu_nms.7z", the error will happened.

Eventhough, I just reinstall the model optimizer pre-requesites and did it again, still got the same error.

[Shubha_R_Intel]

Dear SC Huang,

Your results are believable since  "yolov3_cpu_nms.7z" (wherever it came from) was not tested by the Model Optimizer team. Where did this model come from ?

Thanks,

Shubha

[SC_Huang]

Hi Shubha,

This is our customer's trained model, if you need the original weight and config data,

here's the link https://drive.google.com/file/d/17q8vBDUDU0RJLWvQKUJjCif4uCnA2xYe/view?usp=sharing

our customer had used this model deploy on a NVIDA device before, they are now trying to inference with openVINO on Intel device.

Here is some information about the model:

class name :
    person
    car
    motorbike
    bus
    truck
    bike

data format: NHWC

Input_shape : 1 x 416 x 416 x 3

pb file input/output format:
input tensor name : "Placeholder:0"

output tensor name (predict boxes) :  "concat_9:0"
output tensor name (predict score) :  "mul_6:0"

[Shubha_R_Intel]

Dear SC Huang

 I downloaded your customer's model and tried your model optimizer command myself using OpenVino 2019 R2. I got the same error you did. 

Let's take a look at yolo_v3.json. It looks like this

[
  {
    "id": "TFYOLOV3",
    "match_kind": "general",
    "custom_attributes": {
      "classes": 80,
      "anchors": [10, 13, 16, 30, 33, 23, 30, 61, 62, 45, 59, 119, 116, 90, 156, 198, 373, 326],
      "coords": 4,
      "num": 9,
      "masks":[[6, 7, 8], [3, 4, 5], [0, 1, 2]],
      "entry_points": ["detector/yolo-v3/Reshape", "detector/yolo-v3/Reshape_4", "detector/yolo-v3/Reshape_8"]
    }
  }
]

The error Model Optimizer is giving :

Node with name detector/yolo-v3/Reshape doesn't exist in the graph. 

To fix this you must change the yolo_v3.json file. For starters you can try deleting "detector/yolo-v3/Reshape" from the comma separated list of "entry_points" and see what happens.  My guess is that it will then start complaining about "detector/yolo-v3/Reshape_4"

I tried doing this myself but the *.pb file you provided was too big, what I tried was converting the graphDef into a pbtxt, which succeeded. But the text file was too big to read. What you can do is use Tensorboard or a pbtxt version of your frozen model and replace the entry points in yolo_v3.json with the correct ones. Search through your model and figure out what those entry points are.

Hope it helps,

Thanks,

Shubha

[SC_Huang]

Hi Shubha,

Thanks for replied.

This is the model graph using TensorBoard.

Is it means the entry points are "Placeholder", "OneShotIterator_1", "OneShotIterator" ?

[Shubha_R_Intel]

No those are not the  correct entry points. The entry points to the subgraph are pretty tricky to figure out actually. If you look at the yolo_v3.json you see this:

"entry_points": ["detector/yolo-v3/Reshape", "detector/yolo-v3/Reshape_4", "detector/yolo-v3/Reshape_8"]

Also see the attached image of the standard yolo-v3 model. As you can see the  "entry points" feed into Sigmoid (Activation layer). Hope it helps !

 And it's pretty messy to visualize in Tensorboard so instead I printed out the frozen yolo.pb to a text file and I see this:

node {
  name: "prefix/detector/yolo-v3/split"
  op: "SplitV"
  input: "prefix/detector/yolo-v3/Reshape"
  input: "prefix/detector/yolo-v3/Const"
  input: "prefix/detector/yolo-v3/split/split_dim"

name: "prefix/detector/yolo-v3/split_1"
  op: "SplitV"
  input: "prefix/detector/yolo-v3/Reshape_4"
  input: "prefix/detector/yolo-v3/Const_1"
  input: "prefix/detector/yolo-v3/split_1/split_dim"


 name: "prefix/detector/yolo-v3/split_2"
  op: "SplitV"
  input: "prefix/detector/yolo-v3/Reshape_8"
  input: "prefix/detector/yolo-v3/Const_2"
  input: "prefix/detector/yolo-v3/split_2/split_dim"
  attr {

So you see split, split_1 and split_2 are the major entry points into this model. 

Thanks,

Shubha

[SC_Huang]

It's pretty hard to figure out the entry point...

Attach the pbtxt of the model. But I cannot find out the entry point

[Shubha_R_Intel]

Dear SC Huang,

I agree. It's hard to find the "entry point". And worse, it's a weird name - it is not really an "entry point". But I will certainly take a look. Thanks for attaching your archived model.

Sincerely,

Shubha

[Shubha_R_Intel]

Dear SC Huang,

Thanks for  attaching your model.

In C:\Program Files (x86)\IntelSWTools\openvino_2019.2.275\deployment_tools\model_optimizer\extensions\front\tf\yolo_v3.json (save it to a new local file) change this line :

"entry_points": ["detector/yolo-v3/Reshape", "detector/yolo-v3/Reshape_4", "detector/yolo-v3/Reshape_8"]

To this:

"entry_points": ["yolov3/Reshape", "yolov3/Reshape_4", "yolov3/Reshape_8"]

With this model optimizer command:

python "c:\Program Files (x86)\IntelSWTools\openvino_2019.2.275\deployment_tools\model_optimizer\mo_tf.py" --input_model model.pbtxt --input_model_is_text --tensorflow_use_custom_operations_config yolo_v3.json --log_level DEBUG

And you go pretty far, but will still run into an error:

I0829 15:01:13.659024 15920 infer.py:129] --------------------
I0829 15:01:13.659024 15920 infer.py:130] Partial infer for yolov3/meshgrid_1/mul_1/YoloRegion
I0829 15:01:13.660024 15920 infer.py:131] Op: RegionYolo
E0829 15:01:13.660024 15920 infer.py:180] Cannot infer shapes or values for node "yolov3/meshgrid_1/mul_1/YoloRegion".
E0829 15:01:13.660024 15920 infer.py:181] index 2 is out of bounds for axis 0 with size 2

I did not debug this error, but I have to ask, why are you using this model ? I know your customer trained it on Nvidia. I get it. But it honestly looks different from  https://github.com/mystic123/tensorflow-yolo-v3 . Why not just use what has been validated and tested from http://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_YOLO_From_Tensorflow.html ? I did a diff between the *.pbtxt of the one we support and the one you sent me, and honestly, they look quite different. It seems like your customer might have done more than simply custom-trained because the difference between the already pre-trained and your customer's version is quite significant.

Thanks,

Shubha

[Wiegersma__Aalzen]

I'm getting the same error during conversion (python3 /opt/intel/openvino/deployment_tools/model_optimizer/mo_tf.py --input_model ./frozen_darknet_yolov3_model.pb --tensorflow_use_custom_operations_config ./yolo_v3_tiny.json --input_shape [1,544,544,3] --data_type FP32):

[ ERROR ]  Exception occurred during running replacer "TFYOLOV3" (<class 'extensions.front.tf.YOLO.YoloV3RegionAddon'>): TensorFlow YOLO V3 conversion mechanism was enabled. Entry points "detector/yolo-v3-tiny/Reshape, detector/yolo-v3-tiny/Reshape_4" were provided in the configuration file. Entry points are nodes that feed YOLO Region layers. Node with name detector/yolo-v3-tiny/Reshape doesn't exist in the graph. Refer to documentation about converting YOLO models for more information.

The default Yolo3 network from https://pjreddie.com/darknet/yolo/ converts without problems like in the OpenVINO documentation. I'm using the following repository for training: https://github.com/AlexeyAB/darknet.

It would be nice to have Darknet/Yolo supported natively rather than the conversion using Tensorflow.

[倪__嘉旻]

Shubha R. (Intel) wrote:

No those are not the  correct entry points. The entry points to the subgraph are pretty tricky to figure out actually. If you look at the yolo_v3.json you see this:

"entry_points": ["detector/yolo-v3/Reshape", "detector/yolo-v3/Reshape_4", "detector/yolo-v3/Reshape_8"]

Also see the attached image of the standard yolo-v3 model. As you can see the  "entry points" feed into Sigmoid (Activation layer). Hope it helps !

 And it's pretty messy to visualize in Tensorboard so instead I printed out the frozen yolo.pb to a text file and I see this:

node {
  name: "prefix/detector/yolo-v3/split"
  op: "SplitV"
  input: "prefix/detector/yolo-v3/Reshape"
  input: "prefix/detector/yolo-v3/Const"
  input: "prefix/detector/yolo-v3/split/split_dim"

name: "prefix/detector/yolo-v3/split_1"
  op: "SplitV"
  input: "prefix/detector/yolo-v3/Reshape_4"
  input: "prefix/detector/yolo-v3/Const_1"
  input: "prefix/detector/yolo-v3/split_1/split_dim"


 name: "prefix/detector/yolo-v3/split_2"
  op: "SplitV"
  input: "prefix/detector/yolo-v3/Reshape_8"
  input: "prefix/detector/yolo-v3/Const_2"
  input: "prefix/detector/yolo-v3/split_2/split_dim"
  attr {

So you see split, split_1 and split_2 are the major entry points into this model. 

Thanks,

Shubha

Dear Shubha,

 Can you explain how to open the frozen yolo.pb? l failed to do so..

Thanks,

Kathryn

[ryanyousong]

I also get the same problem, do you solve your problem already? if you solve your problem,please contact me thank you very much! my email is dltbjys1992@163.com