topic Issues converting a model trained with TensorFlow (Keras H5) format to IR in Intel® Distribution of OpenVINO™ Toolkit

Issues converting a model trained with TensorFlow (Keras H5) format to IR

mvasquez — Fri, 06 May 2022 20:35:13 GMT

Hello, I'm following these instructions in order to covert a Keras H5 model to IR format:
https://docs.openvino.ai/latest/openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow.html#keras-h5
The trained H5 model is working great, but I need to use it in a environment with OpenVINO 2021.4.

1) I serialized the H5 model into the SavedFormat format, and I get an structure like this:

# ls model/
assets saved_model.pb variables

2) I run the mo script:

# mo --saved_model_dir model/

and I get this output:

Model Optimizer arguments:
Common parameters:
- Path to the Input Model: None
- Path for generated IR: /opt/intel/openvino_2021.4.689/deployment_tools/model_optimizer/.
- IR output name: saved_model
- Log level: ERROR
- Batch: Not specified, inherited from the model
- Input layers: Not specified, inherited from the model
- Output layers: Not specified, inherited from the model
- Input shapes: Not specified, inherited from the model
- Source layout: Not specified
- Target layout: Not specified
- Layout: Not specified
- Mean values: Not specified
- Scale values: Not specified
- Scale factor: Not specified
- Precision of IR: FP32
- Enable fusing: True
- User transformations: Not specified
- Reverse input channels: False
- Enable IR generation for fixed input shape: False
- Use the transformations config file: None
Advanced parameters:
- Force the usage of legacy Frontend of Model Optimizer for model conversion into IR: False
- Force the usage of new Frontend of Model Optimizer for model conversion into IR: False
TensorFlow specific parameters:
- Input model in text protobuf format: False
- Path to model dump for TensorBoard: None
- List of shared libraries with TensorFlow custom layers implementation: None
- Update the configuration file with input/output node names: None
- Use configuration file used to generate the model with Object Detection API: None
- Use the config file: None
OpenVINO runtime found in: /home/openvino/.local/lib/python3.6/site-packages/openvino
OpenVINO runtime version: 2022.1.0-7019-cdb9bec7210-releases/2022/1
Model Optimizer version: 2022.1.0-7019-cdb9bec7210-releases/2022/1
[ WARNING ] 
Detected not satisfied dependencies:
networkx: installed: 2.5.1, required: ~= 2.6
fastjsonschema: not installed, required: ~= 2.15.1

Please install required versions of components or run pip installation
pip install openvino-dev[tensorflow]
[ WARNING ] The model contains input(s) with partially defined shapes: name="conv2d_input" shape="[-1, 30, 30, 3]". Starting from the 2022.1 release the Model Optimizer can generate an IR with partially defined input shapes ("-1" dimension in the TensorFlow model or dimension with string value in the ONNX model). Some of the OpenVINO plugins require model input shapes to be static, so you should call "reshape" method in the Inference Engine and specify static input shapes. For optimal performance, it is still recommended to update input shapes with fixed ones using "--input" or "--input_shape" command-line parameters.
[ SUCCESS ] Generated IR version 11 model.
[ SUCCESS ] XML file: /opt/intel/openvino_2021.4.689/deployment_tools/model_optimizer/saved_model.xml
[ SUCCESS ] BIN file: /opt/intel/openvino_2021.4.689/deployment_tools/model_optimizer/saved_model.bin
[ SUCCESS ] Total execution time: 3.62 seconds. 
[ SUCCESS ] Memory consumed: 372 MB. 
It's been a while, check for a new version of Intel(R) Distribution of OpenVINO(TM) toolkit here https://software.intel.com/content/www/us/en/develop/tools/openvino-toolkit/download.html?cid=other&source=prod&campid=ww_2022_bu_IOTG_OpenVINO-2022-1&content=upg_all&medium=organic or on the GitHub*
[ INFO ] The model was converted to IR v11, the latest model format that corresponds to the source DL framework input/output format. While IR v11 is backwards compatible with OpenVINO Inference Engine API v1.0, please use API v2.0 (as of 2022.1) to take advantage of the latest improvements in IR v11.
Find more information about API v2.0 and IR v11 at https://docs.openvino.ai

3) But, when I execute the converted IR model with OpenVINO I get this error:

Unknown model format! Cannot find reader for model format: xml and read the model: ./model.xml. Please check that reader library exists in your PATH.
ia_video_ingestion_rsd | Traceback (most recent call last):
ia_video_ingestion_rsd | File "udf.pyx", line 376, in udf.load_udf
ia_video_ingestion_rsd | File "/app/app.py", line 53, in __init__
ia_video_ingestion_rsd | self.neural_net = self.ie_core.read_network(model=model_xml, weights=model_bin)
ia_video_ingestion_rsd | File "ie_api.pyx", line 326, in openvino.inference_engine.ie_api.IECore.read_network
ia_video_ingestion_rsd | File "ie_api.pyx", line 351, in openvino.inference_engine.ie_api.IECore.read_network
ia_video_ingestion_rsd | RuntimeError: Unknown model format! Cannot find reader for model format: xml and read the model: ./model.xml. Please check that reader library exists in your PATH.

Do you have any idea what is the correct way to convert from Keras H5 to IR format?

Note: I also tried with the version 2022 of OpenVINO, no luck.

Regards,

Miguel

Re: Issues converting a model trained with TensorFlow (Keras H5) format to IR

Hairul_Intel — Mon, 09 May 2022 03:21:21 GMT

Hi Miguel,

Thank you for reaching out to us.

Based on your error log, the model was converted by OpenVINO 2022.1 Model Optimizer instead of OpenVINO 2021.4 Model Optimizer.

For your information, Inference Engine of OpenVINO™ Toolkit 2021.4 is unable to read and load model IRv11.

Please ensure that you have initialized the environments for OpenVINO™ Toolkit 2021.4 by running the following script before converting the model:

source /opt/intel/openvino_2021/bin/setupvars.sh

Regards,

Hairul

Re: Issues converting a model trained with TensorFlow (Keras H5) format to IR

mvasquez — Mon, 09 May 2022 14:48:05 GMT

Hi Hairul, thanks for the response. I've repeated the same procedure in OpenVino 2021.4, but executing previously the script:

/opt/intel/openvino_2021.4.689/bin/setup_vars.sh

Now I'm getting this error, do you know how to perform correctly this convertion?

/opt/intel/openvino_2021.4.689/deployment_tools/model_optimizer$ python3 mo.py --saved_model_dir ~/Downloads/model
[ WARNING ] Telemetry will not be sent as TID is not specified.
Model Optimizer arguments:
Common parameters:
- Path to the Input Model: None
- Path for generated IR: /opt/intel/openvino_2021.4.689/deployment_tools/model_optimizer/.
- IR output name: saved_model
- Log level: ERROR
- Batch: Not specified, inherited from the model
- Input layers: Not specified, inherited from the model
- Output layers: Not specified, inherited from the model
- Input shapes: Not specified, inherited from the model
- Mean values: Not specified
- Scale values: Not specified
- Scale factor: Not specified
- Precision of IR: FP32
- Enable fusing: True
- Enable grouped convolutions fusing: True
- Move mean values to preprocess section: None
- Reverse input channels: False
TensorFlow specific parameters:
- Input model in text protobuf format: False
- Path to model dump for TensorBoard: None
- List of shared libraries with TensorFlow custom layers implementation: None
- Update the configuration file with input/output node names: None
- Use configuration file used to generate the model with Object Detection API: None
- Use the config file: None
- Inference Engine found in: /opt/intel/openvino_2021.4.689/python/python3.6/openvino
Inference Engine version: 2021.4.1-3926-14e67d86634-releases/2021/4
Model Optimizer version: 2021.4.1-3926-14e67d86634-releases/2021/4
[ WARNING ] Telemetry will not be sent as TID is not specified.
2022-05-09 11:28:16.836810: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: :/opt/intel/openvino_2021.4.689/deployment_tools/model_optimizer/mo/utils/../../../inference_engine/lib/intel64:/opt/intel/openvino_2021.4.689/deployment_tools/model_optimizer/mo/utils/../../../inference_engine/external/tbb/lib:/opt/intel/openvino_2021.4.689/deployment_tools/model_optimizer/mo/utils/../../../ngraph/lib
2022-05-09 11:28:16.836832: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
/usr/local/lib/python3.6/dist-packages/tensorflow/python/autograph/impl/api.py:22: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
2022-05-09 11:28:18.007461: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2022-05-09 11:28:18.007617: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: :/opt/intel/openvino_2021.4.689/deployment_tools/model_optimizer/mo/utils/../../../inference_engine/lib/intel64:/opt/intel/openvino_2021.4.689/deployment_tools/model_optimizer/mo/utils/../../../inference_engine/external/tbb/lib:/opt/intel/openvino_2021.4.689/deployment_tools/model_optimizer/mo/utils/../../../ngraph/lib
2022-05-09 11:28:18.007630: W tensorflow/stream_executor/cuda/cuda_driver.cc:326] failed call to cuInit: UNKNOWN ERROR (303)
2022-05-09 11:28:18.007642: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (openvino-ATC-8010): /proc/driver/nvidia/version does not exist
2022-05-09 11:28:18.007758: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-05-09 11:28:18.008575: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2022-05-09 11:28:18.265428: I tensorflow/core/grappler/devices.cc:69] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 0
2022-05-09 11:28:18.265514: I tensorflow/core/grappler/clusters/single_machine.cc:356] Starting new session
2022-05-09 11:28:18.265730: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2022-05-09 11:28:18.284247: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2400000000 Hz
2022-05-09 11:28:18.286792: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:928] Optimization results for grappler item: graph_to_optimize
function_optimizer: Graph size after: 69 nodes (53), 92 edges (76), time = 1.251ms.
function_optimizer: function_optimizer did nothing. time = 0.025ms.

[ ERROR ] Shape [-1 30 30 3] is not fully defined for output 0 of "conv2d_input". Use --input_shape with positive integers to override model input shapes.
[ ERROR ] Cannot infer shapes or values for node "conv2d_input".
[ ERROR ] Not all output shapes were inferred or fully defined for node "conv2d_input". 
For more information please refer to Model Optimizer FAQ, question #40. (https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html?question=40#question-40)
[ ERROR ] 
[ ERROR ] It can happen due to bug in custom shape infer function <function Parameter.infer at 0x7f3911d1f2f0>.
[ ERROR ] Or because the node inputs have incorrect values/shapes.
[ ERROR ] Or because input shapes are incorrect (embedded to the model or passed via --input_shape).
[ ERROR ] Run Model Optimizer with --log_level=DEBUG for more information.
[ ERROR ] Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.PartialInfer.PartialInfer'>): Stopped shape/value propagation at "conv2d_input" node. 
For more information please refer to Model Optimizer FAQ, question #38. (https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html?question=38#question-38)

Re:Issues converting a model trained with TensorFlow (Keras H5) format to IR

Hairul_Intel — Tue, 10 May 2022 04:01:33 GMT

Hi Miguel,

Please ensure you have installed the requirements for TensorFlow2 dependencies as mentioned here.

On another note, you can try running the following script when converting your model:

python3 mo_tf.py --saved_model_dir model --batch 1

Additionally, please share more details regarding your model, is it a custom or pre-trained model, the topology, source repository, etc., for further investigation.

Regards,

Hairul

Re: Re:Issues converting a model trained with TensorFlow (Keras H5) format to IR

mvasquez — Tue, 10 May 2022 14:04:20 GMT

Hi Hairul, in order to convert correctly the model I had to add the input_shape parameter:

python3 mo --saved_model_dir ~/Downloads/model --input_shape [1,30,30,3]

The model had been trained with 30x30 images, but not sure what do mean the numbers 1 and 3.

Thanks for the support,

Miguel

Re:Issues converting a model trained with TensorFlow (Keras H5) format to IR

Hairul_Intel — Wed, 11 May 2022 03:18:36 GMT

Hi Miguel,

Glad to know that your issue is resolved.

For sharing purposes, the input shape values are in the order of [N,H,W,C] for TensorFlow models. The meaning for each letter are as follows:

N: number of images in the batch
H: height of the image
W: width of the image
C: number of channels of the image (ex: 3 for RGB, 1 for grayscale…)

This thread will no longer be monitored since this issue has been resolved. If you need any additional information from Intel, please submit a new question.

Regards,

Hairul