instruction for converting yolov4 to IR and retraining

[TarunM]

I am able to download yolov4 using model_downloader. It has weights and some python scripts. How can I use these scripts to convert yolov4 to IR.

I will also need to convert yolov4-tiny . Please provide instructions for that also.

Also, I will need to retrain these models on my customer data. How can I do that. If you can point me to the yolov4 version of tensorflow which you used ; i will retrain using that and convert.

[Vladimir_Dudnik]

@TarunM Support for yolo-v4-tiny-tf model will be published in Open Model Zoo for the 2021.4 release, which is coming soon. Until that, you may try to play with yolo-v4-tiny-tf model from OMZ github (need to switch to `develop` branch).

Regarding instructions on yolo-v4-tf model conversion, is not it right there, in model description? And the link to original model repository, where you should be able to retrain it, is also specified in model description. I'm a bit surprised, how you were able to find and download OMZ model without looking at its description?

Download a Model and Convert it into Inference Engine Format

You can download models and if necessary convert them into Inference Engine format using the Model Downloader and other automation tools as shown in the examples below.

An example of using the Model Downloader:

content_copy

python3 <omz_dir>/tools/downloader/downloader.py --name <model_name>

An example of using the Model Converter:

content_copy

python3 <omz_dir>/tools/downloader/converter.py --name <model_name>

[IntelSupport]

Hi TarunM,

You can convert any OpenVINO models into Intermediate Representation (IR) using Model Optimizer scripts or converter script. Refer to this Converting a Model to Intermediate Representation (IR) documentation for the Model Optimizer and Model converter usage for the instructions.

Meanwhile, for the latest version OpenVINO 2021.3, only yolo-v4-tf is available.

Regards,

Aznie

[TarunMishra]

Following the instructions I downloaded the tiny yolo4

python downloader.py --name yolo-v4-tf -o "C:\Users\mypc\Documents\Desktop\yolov4"

The downloaded directory structure looks as in attached image.

Now its is unclear what to do with it.

Should I use omz/model_optimizer or the script came with this downloads?

Looks like the downloaded weights are yolo but not yolo-tiny.

--------------UPDATE--------------------

I am able to convert, yolov4.weight into hd5 format. Which script should i use now to convert it into IR??

[Vladimir_Dudnik]

Not sure if you had a chance to look at my message above:

Support for yolo-v4-tiny-tf model will be published in Open Model Zoo for the 2021.4 release, which is coming soon. Until that, you may try to play with yolo-v4-tiny-tf model from OMZ github (need to switch to `develop` branch).

An example of using the Model Downloader:

python3 <omz_dir>/tools/downloader/downloader.py --name <model_name>

An example of using the Model Converter:

python3 <omz_dir>/tools/downloader/converter.py --name <model_name>

[TarunMishra]

I cloned develop branch, and I was able to download yolo-v4-tiny-tf

But got error while converting:-

C:\Users\tarunmis\Downloads\open_model_zoo-develop\tools\downloader>python converter.py --name yolo-v4-tiny-tf
========== Running pre-convert script for yolo-v4-tiny-tf
Pre-convert command: C:\Users\tarunmis\AppData\Local\Programs\Python\Python37\python.exe -- C:\Users\tarunmis\Downloads\open_model_zoo-develop\models\public\yolo-v4-tiny-tf\pre-convert.py -- C:\Users\tarunmis\Downloads\open_model_zoo-develop\tools\downloader\public\yolo-v4-tiny-tf C:\Users\tarunmis\Downloads\open_model_zoo-develop\tools\downloader\public\yolo-v4-tiny-tf

C:\Users\tarunmis\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\framework\dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
C:\Users\tarunmis\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\framework\dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
C:\Users\tarunmis\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\framework\dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
C:\Users\tarunmis\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\framework\dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
C:\Users\tarunmis\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\framework\dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
C:\Users\tarunmis\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\framework\dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
Traceback (most recent call last):
File "C:\Users\tarunmis\Downloads\open_model_zoo-develop\tools\downloader\public\yolo-v4-tiny-tf\keras-YOLOv3-model-set\tools\model_converter\convert.py", line 13, in <module>
import tensorflow as tf
File "C:\Users\tarunmis\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\__init__.py", line 28, in <module>
from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import
File "C:\Users\tarunmis\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\__init__.py", line 83, in <module>
from tensorflow.python import keras
File "C:\Users\tarunmis\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\keras\__init__.py", line 26, in <module>
from tensorflow.python.keras import activations
File "C:\Users\tarunmis\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\keras\activations.py", line 24, in <module>
from tensorflow.python.keras.utils.generic_utils import deserialize_keras_object
File "C:\Users\tarunmis\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\keras\utils\__init__.py", line 39, in <module>
from tensorflow.python.keras.utils.multi_gpu_utils import multi_gpu_model
File "C:\Users\tarunmis\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\keras\utils\multi_gpu_utils.py", line 22, in <module>
from tensorflow.python.keras.engine.training import Model
File "C:\Users\tarunmis\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\keras\engine\training.py", line 40, in <module>
from tensorflow.python.keras.engine import network
File "C:\Users\tarunmis\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\keras\engine\network.py", line 39, in <module>
from tensorflow.python.keras import saving
File "C:\Users\tarunmis\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\keras\saving\__init__.py", line 33, in <module>
from tensorflow.python.keras.saving.saved_model import export_saved_model
ImportError: cannot import name 'export_saved_model' from 'tensorflow.python.keras.saving.saved_model' (C:\Users\tarunmis\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\keras\saving\saved_model\__init__.py)
Traceback (most recent call last):
File "C:\Users\tarunmis\Downloads\open_model_zoo-develop\models\public\yolo-v4-tiny-tf\pre-convert.py", line 46, in <module>
main()
File "C:\Users\tarunmis\Downloads\open_model_zoo-develop\models\public\yolo-v4-tiny-tf\pre-convert.py", line 36, in main
], check=True)
File "C:\Users\tarunmis\AppData\Local\Programs\Python\Python37\lib\subprocess.py", line 512, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['C:\\Users\\tarunmis\\AppData\\Local\\Programs\\Python\\Python37\\python.exe', '--', 'C:\\Users\\tarunmis\\Downloads\\open_model_zoo-develop\\tools\\downloader\\public\\yolo-v4-tiny-tf\\keras-YOLOv3-model-set\\tools\\model_converter\\convert.py', 'C:\\Users\\tarunmis\\Downloads\\open_model_zoo-develop\\tools\\downloader\\public\\yolo-v4-tiny-tf\\keras-YOLOv3-model-set\\cfg\\yolov4-tiny.cfg', 'C:\\Users\\tarunmis\\Downloads\\open_model_zoo-develop\\tools\\downloader\\public\\yolo-v4-tiny-tf\\yolov4-tiny.weights', 'C:\\Users\\tarunmis\\Downloads\\open_model_zoo-develop\\tools\\downloader\\public\\yolo-v4-tiny-tf\\yolo-v4-tiny.h5', '--yolo4_reorder']' returned non-zero exit status 1.

FAILED:
yolo-v4-tiny-tf

[Vladimir_Dudnik]

@TarunMishra that is expected, because Open Model Zoo develop branch rely on changes in OpenVINO Python tools distributions (they will be available through PyPi). But it still possible to convert yolo-v4-tiny-tf model with using Model Optimizer from OpenVINO 2021.3 release.

There is how I did that, I've run OMZ converter.py --name yolo-v4-tiny-tf and it failed similar to your, but at console output you may see actual Model Optimizer command line parameters which are needed, so I just run MO directly from Python environment, I've created for OpenVINO 2021.3 release, and was able to successfully convert model to IR

(py3.7-openvino) C:\dev\build\open-model-zoo>python "c:\Program Files (x86)\IntelSWTools\openvino_2021.3.394\deployment_tools\model_optimizer\mo.py" --framework=tf --data_type=FP16 --output_dir=C:\temp\public\yolo-v4-tiny-tf\FP16 --model_name=yolo-v4-tiny-tf --input_shape=[1,416,416,3] --input=image_input --scale_values=image_input[255] --reverse_input_channels --input_model=C:\temp\public\yolo-v4-tiny-tf/yolo-v4-tiny.pb
Model Optimizer arguments:
Common parameters:
        - Path to the Input Model:      C:\temp\public\yolo-v4-tiny-tf/yolo-v4-tiny.pb
        - Path for generated IR:        C:\temp\public\yolo-v4-tiny-tf\FP16
        - IR output name:       yolo-v4-tiny-tf
        - Log level:    ERROR
        - Batch:        Not specified, inherited from the model
        - Input layers:         image_input
        - Output layers:        Not specified, inherited from the model
        - Input shapes:         [1,416,416,3]
        - Mean values:  Not specified
        - Scale values:         image_input[255]
        - Scale factor:         Not specified
        - Precision of IR:      FP16
        - Enable fusing:        True
        - Enable grouped convolutions fusing:   True
        - Move mean values to preprocess section:       None
        - Reverse input channels:       True
TensorFlow specific parameters:
        - Input model in text protobuf format:  False
        - Path to model dump for TensorBoard:   None
        - List of shared libraries with TensorFlow custom layers implementation:        None
        - Update the configuration file with input/output node names:   None
        - Use configuration file used to generate the model with Object Detection API:  None
        - Use the config file:  None
        - Inference Engine found in:    C:\Program Files (x86)\IntelSWTools\openvino_2021.3.394\python\python3.7\openvino
Inference Engine version:       2.1.2021.3.0-2787-60059f2c755-releases/2021/3
Model Optimizer version:            2021.3.0-2787-60059f2c755-releases/2021/3
C:\Miniconda3\envs\py3.7-openvino\lib\site-packages\urllib3\util\selectors.py:14: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working
  from collections import namedtuple, Mapping
C:\Miniconda3\envs\py3.7-openvino\lib\site-packages\urllib3\_collections.py:2: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working
  from collections import Mapping, MutableMapping
2021-05-28 01:54:53.478901: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'cudart64_101.dll'; dlerror: cudart64_101.dll not found
2021-05-28 01:54:53.483862: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
[ SUCCESS ] Generated IR version 10 model.
[ SUCCESS ] XML file: C:\temp\public\yolo-v4-tiny-tf\FP16\yolo-v4-tiny-tf.xml
[ SUCCESS ] BIN file: C:\temp\public\yolo-v4-tiny-tf\FP16\yolo-v4-tiny-tf.bin
[ SUCCESS ] Total execution time: 10.94 seconds.

(py3.7-openvino) C:\dev\build\open-model-zoo>

[TarunM]

You are running it on .pb file.

When i downloaded using:- 

python downloader.py --name yolo-v4-tiny-tf

I got .weight files and bunch of other scripts.How to get .pb file on which you are runing.

[Vladimir_Dudnik]

.pb file was created after I call OMZ converter.py script, although it failed on MO call, but it was successfully converted model to .pb file. Please check if you installed all required Python dependencies, according to instructions.

[IntelSupport · Accepted solution]

Hi TarunM,

The .pb file is generated when you are running converter.py script even though you are getting the error. Then, use the model optimizer to convert the model into IR. Here is the command that I used:

Python mo.py --framework=tf --data_type=FP16 "--output_dir "<OUTPUT_DIR>" --model_name=yolo-v4-tiny-tf --input_shape=[1,416,416,3] --input=image_input --scale_values=image_input[255] --reverse_input_channels "--input_model "<INSTALL_DIR>\open_model_zoo\tools\downloader\public\yolo-v4-tiny-tf/yolo-v4-tiny.pb"

Regards,

Aznie

[IntelSupport]

Hi TarunM,

This thread will no longer be monitored since this issue has been resolved. If you need any additional information from Intel, please submit a new question.

Regards,

Aznie