unable to convert custom tf non-frozen model, or create frozen model

[Portomania]

I've been working on a custom model using tensorflow that I have made into a SavedModel format and am now trying to convert using the mo_tf.py script. 

I am running: "sudo python3.6 mo_tf.py --saved_model_dir ~/Downloads/modelDirtest2 --output_dir ~/Downloads/ --log_level DEBUG"

which ultimately returns:

"mo.utils.error.Error: Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.load.tf.loader.TFLoader'>): Unexpected exception happened during extracting attributes for node RMSprop/lstm_11/lstm_cell_11/bias/rms/Read/ReadVariableOp.
Original exception message: 'ascii' codec can't decode byte 0xdf in position 1: ordinal not in range(128)"

I believe the issue may have something to do with the fact the model was trained and saved in a tensorflow 2.x environment, and now I'm trying to use it in a tensorflow 1.x environment to allow Openvino to function. Does it look like this is the issue? If so, i there a way to tell what part of model isn't compatible with tensorflow 1.x? Is the lstm_cell layer no longer usable?

Due to this, I also tried to freeze the model using what was directed in; https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow.html#freeze-the-tensorflow-model :

from tensorflow.python.framework import graph_io
frozen = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ["output_nodes_here"])
graph_io.write_graph(frozen, './', 'inference_graph.pb', as_text=False)

Since tf.session() in 2.x is deprecated, I then tried:

import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()
from tensorflow.python.framework import graph_io
sess=tf.Session()
frozen = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ["output_nodes_here"])
graph_io.write_graph(frozen, './', 'inference_graph.pb', as_text=False)

This gave me a frozen inference graph file, and now I am unable to determine what output operations I should put from my model to freeze it correctly. Running tensorboard shows me all the OpNodes, but I can't tell which are the outputs. Is there a more updated way to freeze graphs of custom models, or is there a way to determine what output operations must be frozen?

Thanks

[Iffa_Intel · Accepted solution]

Greetings,

Your Tensorflow model's version that you are using need to match with the model that you use to train.

For instance if you are using TF 1.14, ensure to use the same version of trained model of TF 1.14 and then convert it to be use with OpenVino.

You can refer here for a much more detailed informations including step by step instructions:

https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow.html

There are also infos regarding frozen and non-frozen models in there.

Sincerely,

Iffa

[Portomania]

Thanks for getting back to me. 

I've now set up a Ubuntu VM with tensorflow 1.14.0, and checked that OpenVINO accepted this version. Then I brought my model training over and am now trying to save my model. The way to save my model has changed so I'm using "tf.saved_model.save(pretrained_model, save_path).

However now the Traceback error is this: 

File "model.py", line 249, in nn_train
    tf.saved_model.save(model, filename)
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/saved_model/save.py", line 822, in save
    meta_graph_def, saveable_view, signatures)
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/saved_model/save.py", line 510, in _fill_meta_graph_def
    object_map, resource_map, asset_info = saveable_view.map_resources()
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/saved_model/save.py", line 245, in map_resources
    tensor_util.constant_value(capture))
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 246, in constant
    allow_broadcast=True)
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 284, in _constant_impl
    allow_broadcast=allow_broadcast))
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py", line 454, in make_tensor_proto
    raise ValueError("None values not supported.")
ValueError: None values not supported.

It seems to think that my model is empty, however running model.summary() tells me that all of my layers are definitely being trained. Is there a different method to save TF 1.x models that I'm not trying?

-- Port

[Portomania]

Thanks for getting back to me. 

I've now set up a Ubuntu VM with tensorflow 1.14.0, and checked that OpenVINO accepted this version. Then I brought my model training over and am now trying to save my model. The way to save my model has changed so I'm using "tf.saved_model.save(pretrained_model, save_path).

However now the Traceback error is this: 

File "model.py", line 249, in nn_train
    tf.saved_model.save(model, filename)
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/saved_model/save.py", line 822, in save
    meta_graph_def, saveable_view, signatures)
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/saved_model/save.py", line 510, in _fill_meta_graph_def
    object_map, resource_map, asset_info = saveable_view.map_resources()
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/saved_model/save.py", line 245, in map_resources
    tensor_util.constant_value(capture))
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 246, in constant
    allow_broadcast=True)
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 284, in _constant_impl
    allow_broadcast=allow_broadcast))
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py", line 454, in make_tensor_proto
    raise ValueError("None values not supported.")
ValueError: None values not supported.

It seems to think that my model is empty, however running model.summary() tells me that all of my layers are definitely being trained. Is there a different method to save TF 1.x models that I'm not trying?

-- Port

[Portomania]

Thanks for getting back to me. 

I've now set up a Ubuntu VM with tensorflow 1.14.0, and checked that OpenVINO accepted this version. Then I brought my model training over and am now trying to save my model. The way to save my model has changed so I'm using "tf.saved_model.save(pretrained_model, save_path).

However now the Traceback error is this: 

File "model.py", line 249, in nn_train
    tf.saved_model.save(model, filename)
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/saved_model/save.py", line 822, in save
    meta_graph_def, saveable_view, signatures)
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/saved_model/save.py", line 510, in _fill_meta_graph_def
    object_map, resource_map, asset_info = saveable_view.map_resources()
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/saved_model/save.py", line 245, in map_resources
    tensor_util.constant_value(capture))
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 246, in constant
    allow_broadcast=True)
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 284, in _constant_impl
    allow_broadcast=allow_broadcast))
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py", line 454, in make_tensor_proto
    raise ValueError("None values not supported.")
ValueError: None values not supported.

It seems to think that my model is empty, however running model.summary() tells me that all of my layers are definitely being trained. Is there a different method to save TF 1.x models that I'm not trying?

I've been having issues posting this reply.

-- Port

[Iffa_Intel]

There are three ways to store non-frozen TensorFlow models and load them to the Model Optimizer:

1.Checkpoint

2.MetaGraph

3.SavedModel

Each of them have a few steps be done which you can refer here(previous version 2020.3):

https://docs.openvinotoolkit.org/2020.3/_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow.html#tensorflow_specific_conversion_params

Also, please take note of the version of OpenVino that you are using (latest is 2020.4)

Here is the latest reference: https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow.html

If you are using latest Openvino 2020.4, it is recommended to use Tensorflow 2.x where it officially supports two model formats: SavedModel and Keras H5 (or HDF5).

If you are planning to migrate here is some infos: https://www.tensorflow.org/guide/migrate

Sincerely,

Iffa

[Portomania]

Okay, I didn't realize there was a more recent version 2020.4 out with tf 2 compatibility. I fully updated OpenVINO and tensorflow, and am running:

"sudo python3.6 mo_tf.py --saved_model_dir ~/Downloads/saved_model --output_dir ~/Downloads/it_worked --input_shape [1,120,20] ".

Now I'm running into issues with one of the operations: "StatefulPartitionedCall". I read that this particular node can have issues in openvino here: https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow.html#freeze-the-tensorflowmodel 

It says that "TensorFlow 2.x SavedModel format has a specific graph due to eager execution. In case of pruning, find custom input nodes in the StatefulPartitionedCall/* subgraph of TensorFlow 2.x SavedModel format."

Could you please go more in depth into how exactly I should be 'pruning' these node's input?

Thanks again --Port

Oh, here is the error, and I know that my input shape is correct,:

Model Optimizer version:  
Progress: [.......             ]  35.71% done[ ERROR ]  Cannot infer shapes or values for node "StatefulPartitionedCall/sequential/lstm/StatefulPartitionedCall/TensorArrayUnstack/TensorListFromTensor".
[ ERROR ]  Tensorflow type 21 not convertible to numpy dtype.
[ ERROR ] 
[ ERROR ]  It can happen due to bug in custom shape infer function <function tf_native_tf_node_infer at 0x7f39ccd08400>.
[ ERROR ]  Or because the node inputs have incorrect values/shapes.
[ ERROR ]  Or because input shapes are incorrect (embedded to the model or passed via --input_shape).
[ ERROR ]  Run Model Optimizer with --log_level=DEBUG for more information.
[ ERROR ]  Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.PartialInfer.PartialInfer'>): Stopped shape/value propagation at "StatefulPartitionedCall/sequential/lstm/StatefulPartitionedCall/TensorArrayUnstack/TensorListFromTensor" node.
 For more information please refer to Model Optimizer FAQ (https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html), question #38.