Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.

Model Optimizer error with Yolo_v3 model

SC_Huang
Beginner
3,330 Views

I was converting a yolo_v3 model with following arguments

sudo python3 mo_tf.py --input_model ~/openvino_models/models/Customer/yolov3_cpu_nms.pb --tensorflow_use_custom_operations_config extensions/front/tf/yolo_v3.json --input_shape=[1,416,416,3] --input=Placeholder:0 --output=concat_9:0,mul_6:0 --data_type=FP32 --log_level=DEBUG

and get error like this

[ ERROR ]  Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.front.input_cut.InputCut'>): Placeholder node "Placeholder" doesn't have input port, but input port 0 was provided. 

How to slove this problem?

For more information, I've attach debug log and frozen tensorflow model file.

Thank you.

0 Kudos
16 Replies
Shubha_R_Intel
Employee
3,330 Views

Dear SC Huang,

you should not have to add --input=Placeholder:0 --output=concat_9:0,mul_6:0  . If you read the Model Optimizer Yolo V3 doc those switches are not needed.

Can you re-try without those switches ?

Thanks kindly,

Shubha

0 Kudos
SC_Huang
Beginner
3,330 Views

Shubha R. (Intel) wrote:

Dear SC Huang,

you should not have to add --input=Placeholder:0 --output=concat_9:0,mul_6:0  . If you read the Model Optimizer Yolo V3 doc those switches are not needed.

Can you re-try without those switches ?

Thanks kindly,

Shubha

Hi Shubha,

I have tried, error occured as following

[ ERROR ]  Exception occurred during running replacer "TFYOLOV3" (<class 'extensions.front.tf.YOLO.YoloV3RegionAddon'>): TensorFlow YOLO V3 conversion mechanism was enabled. Entry points "detector/yolo-v3/Reshape, detector/yolo-v3/Reshape_4, detector/yolo-v3/Reshape_8" were provided in the configuration file. Entry points are nodes that feed YOLO Region layers. Node with name detector/yolo-v3/Reshape doesn't exist in the graph. Refer to documentation about converting YOLO models for more information.

 

This is the command i tried.

sudo python3 mo_tf.py --input_model ~/openvino_models/models/Customer/yolov3_cpu_nms.pb --tensorflow_use_custom_operations_config extensions/front/tf/yolo_v3.json --data_type=FP32

 

0 Kudos
Shubha_R_Intel
Employee
3,330 Views

Dear SC Huang,

OK, I believe you. Let me try it myself and see what happens. Maybe there is a bug.

Sorry for the trouble !

Shubha

 

0 Kudos
Shubha_R_Intel
Employee
3,330 Views

Dear SC Huang

 I just now tried yolo_v3 following exactly the steps outlined in Convert YOLO Model Optimizer Doc and had no issues. Below is the command I used: 

python  "c:\Program Files (x86)\IntelSWTools\openvino_2019.1.148\deployment_tools\model_optimizer\mo_tf.py" --input_model frozen_darknet_yolov3_model.pb --tensorflow_use_custom_operations_config "c:\Program Files (x86)\IntelSWTools\openvino_2019.1.148\deployment_tools\model_optimizer\extensions\front\tf\yolo_v3.json" --batch 1

And I used OpenVino 2019R1.1.

My guess is that you didn't install Model Optimizer pre-requisites.

Thanks,

Shubha

0 Kudos
SC_Huang
Beginner
3,330 Views

Shubha R. (Intel) wrote:

Dear SC Huang

 I just now tried yolo_v3 following exactly the steps outlined in Convert YOLO Model Optimizer Doc and had no issues. Below is the command I used: 

python  "c:\Program Files (x86)\IntelSWTools\openvino_2019.1.148\deployment_tools\model_optimizer\mo_tf.py" --input_model frozen_darknet_yolov3_model.pb --tensorflow_use_custom_operations_config "c:\Program Files (x86)\IntelSWTools\openvino_2019.1.148\deployment_tools\model_optimizer\extensions\front\tf\yolo_v3.json" --batch 1

And I used OpenVino 2019R1.1.

My guess is that you didn't install Model Optimizer pre-requisites.

Thanks,

Shubha

 

If you were using model from the https://github.com/mystic123/tensorflow-yolo-v3 , I've succesfully done with that before by following steps from your websites.

But when it comes to my Yolo-v3 model as attachment "yolov3_cpu_nms.7z", the error will happened.

Eventhough, I just reinstall the model optimizer pre-requesites and did it again, still got the same error.

0 Kudos
Shubha_R_Intel
Employee
3,330 Views

Dear SC Huang,

Your results are believable since  "yolov3_cpu_nms.7z" (wherever it came from) was not tested by the Model Optimizer team. Where did this model come from ?

Thanks,

Shubha

0 Kudos
SC_Huang
Beginner
3,330 Views

Hi Shubha,

This is our customer's trained model, if you need the original weight and config data,

here's the link https://drive.google.com/file/d/17q8vBDUDU0RJLWvQKUJjCif4uCnA2xYe/view?usp=sharing

our customer had used this model deploy on a NVIDA device before, they are now trying to inference with openVINO on Intel device.

Here is some information about the model:

class name :
    person
    car
    motorbike
    bus
    truck
    bike

data format: NHWC

Input_shape : 1 x 416 x 416 x 3

pb file input/output format:
input tensor name : "Placeholder:0"

output tensor name (predict boxes) :  "concat_9:0"
output tensor name (predict score) :  "mul_6:0"

0 Kudos
Shubha_R_Intel
Employee
3,330 Views

Dear SC Huang

 I downloaded your customer's model and tried your model optimizer command myself using OpenVino 2019 R2. I got the same error you did. 

Let's take a look at yolo_v3.json. It looks like this

[
  {
    "id": "TFYOLOV3",
    "match_kind": "general",
    "custom_attributes": {
      "classes": 80,
      "anchors": [10, 13, 16, 30, 33, 23, 30, 61, 62, 45, 59, 119, 116, 90, 156, 198, 373, 326],
      "coords": 4,
      "num": 9,
      "masks":[[6, 7, 8], [3, 4, 5], [0, 1, 2]],
      "entry_points": ["detector/yolo-v3/Reshape", "detector/yolo-v3/Reshape_4", "detector/yolo-v3/Reshape_8"]
    }
  }
]

The error Model Optimizer is giving :

Node with name detector/yolo-v3/Reshape doesn't exist in the graph. 

To fix this you must change the yolo_v3.json file. For starters you can try deleting "detector/yolo-v3/Reshape" from the comma separated list of "entry_points" and see what happens.  My guess is that it will then start complaining about "detector/yolo-v3/Reshape_4"

I tried doing this myself but the *.pb file you provided was too big, what I tried was converting the graphDef into a pbtxt, which succeeded. But the text file was too big to read. What you can do is use Tensorboard or a pbtxt version of your frozen model and replace the entry points in yolo_v3.json with the correct ones. Search through your model and figure out what those entry points are.

Hope it helps,

Thanks,

Shubha

0 Kudos
SC_Huang
Beginner
3,330 Views

Hi Shubha,

Thanks for replied.

This is the model graph using TensorBoard.

Is it means the entry points are "Placeholder", "OneShotIterator_1", "OneShotIterator" ?

graph_run=.png

0 Kudos
Shubha_R_Intel
Employee
3,330 Views

No those are not the  correct entry points. The entry points to the subgraph are pretty tricky to figure out actually. If you look at the yolo_v3.json you see this:

"entry_points": ["detector/yolo-v3/Reshape", "detector/yolo-v3/Reshape_4", "detector/yolo-v3/Reshape_8"]

Also see the attached image of the standard yolo-v3 model. As you can see the  "entry points" feed into Sigmoid (Activation layer). Hope it helps !


 And it's pretty messy to visualize in Tensorboard so instead I printed out the frozen yolo.pb to a text file and I see this:

node {
  name: "prefix/detector/yolo-v3/split"
  op: "SplitV"
  input: "prefix/detector/yolo-v3/Reshape"
  input: "prefix/detector/yolo-v3/Const"
  input: "prefix/detector/yolo-v3/split/split_dim"

name: "prefix/detector/yolo-v3/split_1"
  op: "SplitV"
  input: "prefix/detector/yolo-v3/Reshape_4"
  input: "prefix/detector/yolo-v3/Const_1"
  input: "prefix/detector/yolo-v3/split_1/split_dim"


 name: "prefix/detector/yolo-v3/split_2"
  op: "SplitV"
  input: "prefix/detector/yolo-v3/Reshape_8"
  input: "prefix/detector/yolo-v3/Const_2"
  input: "prefix/detector/yolo-v3/split_2/split_dim"
  attr {

 

yolov3-small.png

So you see split, split_1 and split_2 are the major entry points into this model. 

Thanks,

Shubha

0 Kudos
SC_Huang
Beginner
3,330 Views

It's pretty hard to figure out the entry point...

Attach the pbtxt of the model. But I cannot find out the entry point

0 Kudos
Shubha_R_Intel
Employee
3,330 Views

Dear SC Huang,

I agree. It's hard to find the "entry point". And worse, it's a weird name - it is not really an "entry point". But I will certainly take a look. Thanks for attaching your archived model.

Sincerely,

Shubha

0 Kudos
Shubha_R_Intel
Employee
3,330 Views

Dear SC Huang,

Thanks for  attaching your model.

In C:\Program Files (x86)\IntelSWTools\openvino_2019.2.275\deployment_tools\model_optimizer\extensions\front\tf\yolo_v3.json (save it to a new local file) change this line :

"entry_points": ["detector/yolo-v3/Reshape", "detector/yolo-v3/Reshape_4", "detector/yolo-v3/Reshape_8"]

To this:

"entry_points": ["yolov3/Reshape", "yolov3/Reshape_4", "yolov3/Reshape_8"]

 

With this model optimizer command:

python "c:\Program Files (x86)\IntelSWTools\openvino_2019.2.275\deployment_tools\model_optimizer\mo_tf.py" --input_model model.pbtxt --input_model_is_text --tensorflow_use_custom_operations_config yolo_v3.json --log_level DEBUG

And you go pretty far, but will still run into an error:

I0829 15:01:13.659024 15920 infer.py:129] --------------------
I0829 15:01:13.659024 15920 infer.py:130] Partial infer for yolov3/meshgrid_1/mul_1/YoloRegion
I0829 15:01:13.660024 15920 infer.py:131] Op: RegionYolo
E0829 15:01:13.660024 15920 infer.py:180] Cannot infer shapes or values for node "yolov3/meshgrid_1/mul_1/YoloRegion".
E0829 15:01:13.660024 15920 infer.py:181] index 2 is out of bounds for axis 0 with size 2

I did not debug this error, but I have to ask, why are you using this model ? I know your customer trained it on Nvidia. I get it. But it honestly looks different from  https://github.com/mystic123/tensorflow-yolo-v3 . Why not just use what has been validated and tested from http://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_YOLO_From_Tensorflow.html ? I did a diff between the *.pbtxt of the one we support and the one you sent me, and honestly, they look quite different. It seems like your customer might have done more than simply custom-trained because the difference between the already pre-trained and your customer's version is quite significant.

Thanks,

Shubha

 

0 Kudos
Wiegersma__Aalzen
3,330 Views

I'm getting the same error during conversion (python3 /opt/intel/openvino/deployment_tools/model_optimizer/mo_tf.py --input_model ./frozen_darknet_yolov3_model.pb --tensorflow_use_custom_operations_config ./yolo_v3_tiny.json --input_shape [1,544,544,3] --data_type FP32):

[ ERROR ]  Exception occurred during running replacer "TFYOLOV3" (<class 'extensions.front.tf.YOLO.YoloV3RegionAddon'>): TensorFlow YOLO V3 conversion mechanism was enabled. Entry points "detector/yolo-v3-tiny/Reshape, detector/yolo-v3-tiny/Reshape_4" were provided in the configuration file. Entry points are nodes that feed YOLO Region layers. Node with name detector/yolo-v3-tiny/Reshape doesn't exist in the graph. Refer to documentation about converting YOLO models for more information.

The default Yolo3 network from https://pjreddie.com/darknet/yolo/ converts without problems like in the OpenVINO documentation. I'm using the following repository for training: https://github.com/AlexeyAB/darknet.

 

It would be nice to have Darknet/Yolo supported natively rather than the conversion using Tensorflow.

0 Kudos
倪__嘉旻
Beginner
3,330 Views

Shubha R. (Intel) wrote:

No those are not the  correct entry points. The entry points to the subgraph are pretty tricky to figure out actually. If you look at the yolo_v3.json you see this:

"entry_points": ["detector/yolo-v3/Reshape", "detector/yolo-v3/Reshape_4", "detector/yolo-v3/Reshape_8"]

Also see the attached image of the standard yolo-v3 model. As you can see the  "entry points" feed into Sigmoid (Activation layer). Hope it helps !

 And it's pretty messy to visualize in Tensorboard so instead I printed out the frozen yolo.pb to a text file and I see this:

node {
  name: "prefix/detector/yolo-v3/split"
  op: "SplitV"
  input: "prefix/detector/yolo-v3/Reshape"
  input: "prefix/detector/yolo-v3/Const"
  input: "prefix/detector/yolo-v3/split/split_dim"

name: "prefix/detector/yolo-v3/split_1"
  op: "SplitV"
  input: "prefix/detector/yolo-v3/Reshape_4"
  input: "prefix/detector/yolo-v3/Const_1"
  input: "prefix/detector/yolo-v3/split_1/split_dim"


 name: "prefix/detector/yolo-v3/split_2"
  op: "SplitV"
  input: "prefix/detector/yolo-v3/Reshape_8"
  input: "prefix/detector/yolo-v3/Const_2"
  input: "prefix/detector/yolo-v3/split_2/split_dim"
  attr {

 

yolov3-small.png

So you see split, split_1 and split_2 are the major entry points into this model. 

Thanks,

Shubha

Dear Shubha,

 Can you explain how to open the frozen yolo.pb? l failed to do so..

Thanks,

Kathryn

0 Kudos
ryanyousong
Beginner
3,330 Views

I also get the same problem, do you solve your problem already? if you solve your problem,please contact me thank you very much! my email is dltbjys1992@163.com

0 Kudos
Reply