topic Re: Issue when converting on Yolo V3 Model using Model Optimizer in Intel® Distribution of OpenVINO™ Toolkit

Issue when converting on Yolo V3 Model using Model Optimizer

DarkHorse — Wed, 11 Aug 2021 06:06:16 GMT

Hello Experts,

My customer had tried to convert Yolo V3 models to IR format:

Converted YOLO V3 model to .pb format – Successfully converted.
Error occurred while converting .pb to IR format.

Have you guys seen these error messages before?

Thanks.

Re: Issue when converting on Yolo V3 Model using Model Optimizer

IntelSupport — Thu, 12 Aug 2021 06:10:45 GMT

Hi Darkhouse,

Thanks for reaching out.

Does your Yolo-v3 model is a custom model? Could your share the model for us to test it on our side? Meanwhile, please try to convert the model using the Model Optimizer arguments for OpenVINO yolo-v3-tf model as below:

mo.py --input_shape=[1,416,416,3] --input=input_1 --scale_values=input_1[255] --reverse_input_channels --transformations_config "INSTALL_DIR\openvino_2021.4.582\deployment_tools\open_model_zoo\tools\downloader\public\yolo-v3-tf\yolo-v3.json" --input_model "INSTALL_DIR\openvino_2021.4.582\deployment_tools\open_model_zoo\tools\downloader\public\yolo-v3-tf\yolo-v3.pb"

You also can refer to the model.yml file of the yolo-v3-tf model for the Model Optimizer arguments. The model.yml file can be found in this directory:

INSTALL_DIR\openvino_2021.4.582\deployment_tools\open_model_zoo\models\public\yolo-v3-tf

Regards,

Aznie

Re: Re:Issue when converting on Yolo V3 Model using Model Optimizer

DarkHorse — Thu, 12 Aug 2021 04:09:45 GMT

Hello Azine,

I have no issue when converting the yolo-v3-tf from the Open Model Zoo to IR format.

Yes, the model is customer's own Yolov3 model. Since this is from customer's model, I will send the model to you personally.

I did try to run model optimizer and I am getting these error messages:

C:\Program Files (x86)\IntelSWTools\openvino_2021\deployment_tools\model_optimizer>python mo.py --input_model "C:\Users\allensen\Downloads\Shelfie_Model\frozen_darknet_yolov3_model.pb" --transformations_config "C:\Users\allensen\Downloads\Shelfie_Model\yolo-v3.json" --data_type=FP32 --model_name yolov3_Shelfie_FP32 --output_dir "C:\Users\allensen\Documents\Intel\IR_Models" --input_shape=[1,416,416,3] --scale_values=input_1[255] --reverse_input_channels
C:\Users\allensen\AppData\Local\Programs\Python\Python37\lib\site-packages\numpy\_distributor_init.py:32: UserWarning: loaded more than 1 DLL from .libs:
C:\Users\allensen\AppData\Local\Programs\Python\Python37\lib\site-packages\numpy\.libs\libopenblas.GK7GX5KEQ4F6UYO3P26ULGBQYHGQO7J4.gfortran-win_amd64.dll
C:\Users\allensen\AppData\Local\Programs\Python\Python37\lib\site-packages\numpy\.libs\libopenblas.WCDJNK7YVMPZQ2ME2ZZHJJRJ3JIKNDB7.gfortran-win_amd64.dll
stacklevel=1)
Model Optimizer arguments:
Common parameters:
- Path to the Input Model: C:\Users\allensen\Downloads\Shelfie_Model\frozen_darknet_yolov3_model.pb
- Path for generated IR: C:\Users\allensen\Documents\Intel\IR_Models
- IR output name: yolov3_Shelfie_FP32
- Log level: ERROR
- Batch: Not specified, inherited from the model
- Input layers: Not specified, inherited from the model
- Output layers: Not specified, inherited from the model
- Input shapes: [1,416,416,3]
- Mean values: Not specified
- Scale values: input_1[255]
- Scale factor: Not specified
- Precision of IR: FP32
- Enable fusing: True
- Enable grouped convolutions fusing: True
- Move mean values to preprocess section: None
- Reverse input channels: True
TensorFlow specific parameters:
- Input model in text protobuf format: False
- Path to model dump for TensorBoard: None
- List of shared libraries with TensorFlow custom layers implementation: None
- Update the configuration file with input/output node names: None
- Use configuration file used to generate the model with Object Detection API: None
- Use the config file: None
C:\Users\allensen\AppData\Local\Programs\Python\Python37\lib\site-packages\numpy\_distributor_init.py:32: UserWarning: loaded more than 1 DLL from .libs:
C:\Users\allensen\AppData\Local\Programs\Python\Python37\lib\site-packages\numpy\.libs\libopenblas.GK7GX5KEQ4F6UYO3P26ULGBQYHGQO7J4.gfortran-win_amd64.dll
C:\Users\allensen\AppData\Local\Programs\Python\Python37\lib\site-packages\numpy\.libs\libopenblas.WCDJNK7YVMPZQ2ME2ZZHJJRJ3JIKNDB7.gfortran-win_amd64.dll
stacklevel=1)
- Inference Engine found in: C:\Program Files (x86)\IntelSWTools\openvino_2021\python\python3.7\openvino
Inference Engine version: 2.1.2021.3.0-2787-60059f2c755-releases/2021/3
Model Optimizer version: 2021.3.0-2787-60059f2c755-releases/2021/3
2021-08-12 12:04:53.717179: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found
2021-08-12 12:04:53.735235: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
[ ANALYSIS INFO ] Your model looks like YOLOv3 Model.
To generate the IR, provide TensorFlow YOLOv3 Model to the Model Optimizer with the following parameters:
--input_model <path_to_model>/yolo_v3.pb
--batch 1
--tensorflow_use_custom_operations_config <OPENVINO_INSTALL_DIR>/deployment_tools/model_optimizer/extensions/front/tf/yolo_v3.json
Detailed information about conversion of this model can be found at
https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_YOLO_From_Tensorflow.html
[ ERROR ] Exception occurred during running replacer "TFYOLOV3" (<class 'extensions.front.YOLO.YoloV3RegionAddon'>): TensorFlow YOLO V3 conversion mechanism was enabled. Entry points "conv2d_58/BiasAdd, conv2d_66/BiasAdd, conv2d_74/BiasAdd" were provided in the configuration file. Entry points are nodes that feed YOLO Region layers. Node with name conv2d_58/BiasAdd doesn't exist in the graph. Refer to documentation about converting YOLO models for more information.

Thank you.

Re: Re:Issue when converting on Yolo V3 Model using Model Optimizer

DarkHorse — Thu, 12 Aug 2021 04:26:37 GMT

Hello Aznie,

Some how, I managed to zoom to the error messages related to the .json file:

C:\Program Files (x86)\IntelSWTools\openvino_2021\deployment_tools\model_optimizer>python mo_tf.py --input_model "C:\Users\allensen\Downloads\Shelfie_Model\frozen_darknet_yolov3_model.pb" --transformations_config "C:\Users\allensen\Downloads\Shelfie_Model\yolo-v3.json" --data_type=FP32 --model_name yolov3_Shelfie_FP32 --output_dir "C:\Users\allensen\Documents\Intel\IR_Models" --input_shape=[1,416,416,3] --scale_values=input_1[255] --reverse_input_channel
C:\Users\allensen\AppData\Local\Programs\Python\Python37\lib\site-packages\numpy\_distributor_init.py:32: UserWarning: loaded more than 1 DLL from .libs:
C:\Users\allensen\AppData\Local\Programs\Python\Python37\lib\site-packages\numpy\.libs\libopenblas.GK7GX5KEQ4F6UYO3P26ULGBQYHGQO7J4.gfortran-win_amd64.dll
C:\Users\allensen\AppData\Local\Programs\Python\Python37\lib\site-packages\numpy\.libs\libopenblas.WCDJNK7YVMPZQ2ME2ZZHJJRJ3JIKNDB7.gfortran-win_amd64.dll
stacklevel=1)
Model Optimizer arguments:
Common parameters:
- Path to the Input Model: C:\Users\allensen\Downloads\Shelfie_Model\frozen_darknet_yolov3_model.pb
- Path for generated IR: C:\Users\allensen\Documents\Intel\IR_Models
- IR output name: yolov3_Shelfie_FP32
- Log level: ERROR
- Batch: Not specified, inherited from the model
- Input layers: Not specified, inherited from the model
- Output layers: Not specified, inherited from the model
- Input shapes: [1,416,416,3]
- Mean values: Not specified
- Scale values: input_1[255]
- Scale factor: Not specified
- Precision of IR: FP32
- Enable fusing: True
- Enable grouped convolutions fusing: True
- Move mean values to preprocess section: None
- Reverse input channels: True
TensorFlow specific parameters:
- Input model in text protobuf format: False
- Path to model dump for TensorBoard: None
- List of shared libraries with TensorFlow custom layers implementation: None
- Update the configuration file with input/output node names: None
- Use configuration file used to generate the model with Object Detection API: None
- Use the config file: None
C:\Users\allensen\AppData\Local\Programs\Python\Python37\lib\site-packages\numpy\_distributor_init.py:32: UserWarning: loaded more than 1 DLL from .libs:
C:\Users\allensen\AppData\Local\Programs\Python\Python37\lib\site-packages\numpy\.libs\libopenblas.GK7GX5KEQ4F6UYO3P26ULGBQYHGQO7J4.gfortran-win_amd64.dll
C:\Users\allensen\AppData\Local\Programs\Python\Python37\lib\site-packages\numpy\.libs\libopenblas.WCDJNK7YVMPZQ2ME2ZZHJJRJ3JIKNDB7.gfortran-win_amd64.dll
stacklevel=1)
- Inference Engine found in: C:\Program Files (x86)\IntelSWTools\openvino_2021\python\python3.7\openvino
Inference Engine version: 2.1.2021.3.0-2787-60059f2c755-releases/2021/3
Model Optimizer version: 2021.3.0-2787-60059f2c755-releases/2021/3
2021-08-12 12:15:38.838739: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found
2021-08-12 12:15:38.855693: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
[ ANALYSIS INFO ] Your model looks like YOLOv3 Model.
To generate the IR, provide TensorFlow YOLOv3 Model to the Model Optimizer with the following parameters:
--input_model <path_to_model>/yolo_v3.pb
--batch 1
--tensorflow_use_custom_operations_config <OPENVINO_INSTALL_DIR>/deployment_tools/model_optimizer/extensions/front/tf/yolo_v3.json
Detailed information about conversion of this model can be found at
https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_YOLO_From_Tensorflow.html
[ ERROR ] Exception occurred during running replacer "TFYOLOV3" (<class 'extensions.front.YOLO.YoloV3RegionAddon'>): TensorFlow YOLO V3 conversion mechanism was enabled. Entry points "conv2d_58/BiasAdd, conv2d_66/BiasAdd, conv2d_74/BiasAdd" were provided in the configuration file. Entry points are nodes that feed YOLO Region layers. Node with name conv2d_58/BiasAdd doesn't exist in the graph. Refer to documentation about converting YOLO models for more information.

Can you send me a private message so that I can share the model's files to you?

Thanks.

Re: Issue when converting on Yolo V3 Model using Model Optimizer

IntelSupport — Fri, 13 Aug 2021 03:06:36 GMT

Hi Darkhouse,

The error you got is because the model node name was different that causing the IR cannot point to the correct node (e.g. conv2d_14 = type: Conv2D, name: detector/yolo-v3/Conv_14/Conv2D - without the supposing value: conv2d_14 )

Comparing your yolov-3 model and Open Model Zoo yolo-v3 model, both .json and model topology use the same naming convention. Hence, we would advise you to follow a similar practice as Open Model Zoo yolo-v3 model to avoid the error from happening again.

I attached the screenshot for both comparisons for a clear view.

Regards,

Aznie

Re: Issue when converting on Yolo V3 Model using Model Optimizer

DarkHorse — Fri, 13 Aug 2021 05:18:09 GMT

Hello Aznie,

Thanks a lot for pointing out the node name mismatch. I have modified to yolo-v3.json file as below:

[
{
"id": "TFYOLOV3",
"match_kind": "general",
"custom_attributes": {
"classes": 43,
"anchors": [0.31,0.75, 0.39,1.51, 0.58,1.04, 0.59,1.89, 0.65,0.60, 0.91,1.61, 1.03,0.99, 1.25,2.03, 1.43,4.16],
"coords": 4,
"num": 9,
"masks":[[6, 7, 8], [3, 4, 5], [0, 1, 2]],
"entry_points": ["detector/yolo-v3/Conv_6/Conv2D", "detector/yolo-v3/Conv_14/Conv2D", "detector/yolo-v3/Conv_22/Conv2D"]
}
}
]

But I bumped into further errors:

C:\Program Files (x86)\IntelSWTools\openvino_2021\deployment_tools\model_optimizer>python mo_tf.py --input_model "C:\Users\allensen\Downloads\Shelfie_Model\frozen_darknet_yolov3_model.pb" --transformations_config "C:\Users\allensen\Downloads\Shelfie_Model\yolo-v3.json" --data_type=FP32 --model_name yolov3_Shelfie_FP32 --output_dir "C:\Users\allensen\Documents\Intel\IR_Models" --input_shape=[1,416,416,3] --scale_values=input_1[255] --reverse_input_channels
C:\Users\allensen\AppData\Local\Programs\Python\Python37\lib\site-packages\numpy\_distributor_init.py:32: UserWarning: loaded more than 1 DLL from .libs:
C:\Users\allensen\AppData\Local\Programs\Python\Python37\lib\site-packages\numpy\.libs\libopenblas.GK7GX5KEQ4F6UYO3P26ULGBQYHGQO7J4.gfortran-win_amd64.dll
C:\Users\allensen\AppData\Local\Programs\Python\Python37\lib\site-packages\numpy\.libs\libopenblas.WCDJNK7YVMPZQ2ME2ZZHJJRJ3JIKNDB7.gfortran-win_amd64.dll
stacklevel=1)
Model Optimizer arguments:
Common parameters:
- Path to the Input Model: C:\Users\allensen\Downloads\Shelfie_Model\frozen_darknet_yolov3_model.pb
- Path for generated IR: C:\Users\allensen\Documents\Intel\IR_Models
- IR output name: yolov3_Shelfie_FP32
- Log level: ERROR
- Batch: Not specified, inherited from the model
- Input layers: Not specified, inherited from the model
- Output layers: Not specified, inherited from the model
- Input shapes: [1,416,416,3]
- Mean values: Not specified
- Scale values: input_1[255]
- Scale factor: Not specified
- Precision of IR: FP32
- Enable fusing: True
- Enable grouped convolutions fusing: True
- Move mean values to preprocess section: None
- Reverse input channels: True
TensorFlow specific parameters:
- Input model in text protobuf format: False
- Path to model dump for TensorBoard: None
- List of shared libraries with TensorFlow custom layers implementation: None
- Update the configuration file with input/output node names: None
- Use configuration file used to generate the model with Object Detection API: None
- Use the config file: None
C:\Users\allensen\AppData\Local\Programs\Python\Python37\lib\site-packages\numpy\_distributor_init.py:32: UserWarning: loaded more than 1 DLL from .libs:
C:\Users\allensen\AppData\Local\Programs\Python\Python37\lib\site-packages\numpy\.libs\libopenblas.GK7GX5KEQ4F6UYO3P26ULGBQYHGQO7J4.gfortran-win_amd64.dll
C:\Users\allensen\AppData\Local\Programs\Python\Python37\lib\site-packages\numpy\.libs\libopenblas.WCDJNK7YVMPZQ2ME2ZZHJJRJ3JIKNDB7.gfortran-win_amd64.dll
stacklevel=1)
- Inference Engine found in: C:\Program Files (x86)\IntelSWTools\openvino_2021\python\python3.7\openvino
Inference Engine version: 2.1.2021.3.0-2787-60059f2c755-releases/2021/3
Model Optimizer version: 2021.3.0-2787-60059f2c755-releases/2021/3
2021-08-13 13:07:32.521545: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found
2021-08-13 13:07:32.540415: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
[ ERROR ] -------------------------------------------------
[ ERROR ] ----------------- INTERNAL ERROR ----------------
[ ERROR ] Unexpected exception happened.
[ ERROR ] Please contact Model Optimizer developers and forward the following information:
[ ERROR ] Exception occurred during running replacer "REPLACEMENT_ID (<class 'extensions.middle.AddMeanScaleValues.AddMeanScaleValues'>)": Attempt to access node None that not in graph
[ ERROR ] Traceback (most recent call last):
File "C:\Program Files (x86)\IntelSWTools\openvino_2021\deployment_tools\model_optimizer\mo\utils\class_registration.py", line 288, in apply_transform
replacer.find_and_replace_pattern(graph)
File "C:\Program Files (x86)\IntelSWTools\openvino_2021\deployment_tools\model_optimizer\extensions\middle\AddMeanScaleValues.py", line 111, in find_and_replace_pattern
if Node(graph, node_id) not in input_nodes:
File "C:\Program Files (x86)\IntelSWTools\openvino_2021\deployment_tools\model_optimizer\mo\graph\graph.py", line 37, in __init__
assert node in graph, "Attempt to access node {} that not in graph".format(node)
AssertionError: Attempt to access node None that not in graph

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "C:\Program Files (x86)\IntelSWTools\openvino_2021\deployment_tools\model_optimizer\mo\main.py", line 345, in main
ret_code = driver(argv)
File "C:\Program Files (x86)\IntelSWTools\openvino_2021\deployment_tools\model_optimizer\mo\main.py", line 309, in driver
ret_res = emit_ir(prepare_ir(argv), argv)
File "C:\Program Files (x86)\IntelSWTools\openvino_2021\deployment_tools\model_optimizer\mo\main.py", line 252, in prepare_ir
graph = unified_pipeline(argv)
File "C:\Program Files (x86)\IntelSWTools\openvino_2021\deployment_tools\model_optimizer\mo\pipeline\unified.py", line 29, in unified_pipeline
class_registration.ClassType.BACK_REPLACER
File "C:\Program Files (x86)\IntelSWTools\openvino_2021\deployment_tools\model_optimizer\mo\utils\class_registration.py", line 340, in apply_replacements
apply_replacements_list(graph, replacers_order)
File "C:\Program Files (x86)\IntelSWTools\openvino_2021\deployment_tools\model_optimizer\mo\utils\class_registration.py", line 330, in apply_replacements_list
num_transforms=len(replacers_order))
File "C:\Program Files (x86)\IntelSWTools\openvino_2021\deployment_tools\model_optimizer\mo\utils\logger.py", line 124, in wrapper
function(*args, **kwargs)
File "C:\Program Files (x86)\IntelSWTools\openvino_2021\deployment_tools\model_optimizer\mo\utils\class_registration.py", line 318, in apply_transform
)) from err
Exception: Exception occurred during running replacer "REPLACEMENT_ID (<class 'extensions.middle.AddMeanScaleValues.AddMeanScaleValues'>)": Attempt to access node None that not in graph

[ ERROR ] ---------------- END OF BUG REPORT --------------
[ ERROR ] -------------------------------------------------

Thanks,

Re:Issue when converting on Yolo V3 Model using Model Optimizer

IntelSupport — Thu, 19 Aug 2021 08:48:56 GMT

Hi Darkhorse,

Can you share with me the source of your model? Probably you can share the link where you got the model. As for now, we observed the same issue and it might be due to the unsupported operation. However, we are still checking with our developer for better clarification.

Regards,

Aznie

Re: Re:Issue when converting on Yolo V3 Model using Model Optimizer

DarkHorse — Thu, 19 Aug 2021 10:55:58 GMT

Hello Aznie,

You may find the .weights file in the previous share drive link that I shared previously.

Customer is using TF version 1.11.0 to freeze the model.

Do let me know if you need further info on this.

Thanks.

Re:Issue when converting on Yolo V3 Model using Model Optimizer

IntelSupport — Fri, 20 Aug 2021 07:53:34 GMT

Hi Darkhouse,

For us to further investigate the issue, could you provide us with the non-frozen models? You could share the link via private message.

Regards,

Aznie

Re: Re:Issue when converting on Yolo V3 Model using Model Optimizer

DarkHorse — Mon, 23 Aug 2021 13:10:59 GMT

Hello Aznie,

My customer is unable to find .meta, .index, .data and checkpoint.

All he has are: .names, .pb, .cfg, .weights and .json files

Is there any other alternative for this to move forward?

Thanks.

Re: Issue when converting on Yolo V3 Model using Model Optimizer

niluwin — Tue, 24 Aug 2021 11:55:40 GMT

Please follow following process to convert your model:

Freeze your model:

git clone https://github.com/mystic123/tensorflow-yolo-v3.git
cd tensorflow-yolo-v3
git checkout ed60b90
use your class and weights file names
pip3 install tensorflow==1.15
In above step if tensorflow-gpu is installed uninstall it using pip3 uninstall tensorflow-gpu as it is used as default.
Make sure that you have right tf version by checking print(tf.__version__)
Pip3 install pillow
python3 convert_weights_pb.py --class_names your_class.names --data_format NHWC --weights_file your_file.weights
This will create frozen_darknet_yolov3_model.pb

Convert the model

Visualize the model may be in netron.app and identify the 3 stages of model output. For more information refer Overview of YOLOv3 Model Architecture
Modify the entry_points field in yolo-v3.json accordingly.
Intialize openvino environment ( source /opt/intel/openvino_2021/bin/setupvars.sh )
mo_tf.py --input_model frozen_darknet_yolov3_model.pb --transformations_config yolo-v3.json --output_dir <output model folder> --input_shape [1,416,416,3]

These steps are tested with OpenVINO version 2021.4

Re: Issue when converting on Yolo V3 Model using Model Optimizer

DarkHorse — Thu, 26 Aug 2021 00:23:39 GMT

Hello @niluwin ,

Thanks, I verified in my machine and these steps works.

Thanks a lot for the help.

Re:Issue when converting on Yolo V3 Model using Model Optimizer

IntelSupport — Fri, 27 Aug 2021 01:53:32 GMT

Hi Darkhouse,

This thread will no longer be monitored since this issue has been resolved. If you need any additional information from Intel, please submit a new question.

Regards,

Aznie