Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.

Error converting TF2 Retinanet

dopeuser
Novice
917 Views

OpenVino version 2021.2

I am trying to convert the inference model of the tf2_retinanet available here: https://github.com/fizyr/tf-retinanet

Since it is a TF2 model, I am using the SavedModel format. I have attached the model in saved_model.zip. It's a Drive link as the model is too big to upload. https://drive.google.com/file/d/1npneE4gxsI0G16hTtjpJkSvnAEHRAqkM/view?usp=sharing

I used this command to optimize the model:

python "C:\Program Files (x86)\intel\openvino_2021\deployment_tools\model_optimizer\mo_tf.py" --saved_model_dir saved_model --input_shape=[1,600,600,3] --input=input_1 --mean_values=[103.939,116.779,123.68] --output=Identity,Identity_1,Identity_3 --log_level=DEBUG

 

This is the last part of the log:

[ 2021-02-07 20:13:15,840 ] [ DEBUG ] [ infer:130 ] Partial infer for StatefulPartitionedCall/retinanet-bbox/classification/concat
[ 2021-02-07 20:13:15,863 ] [ DEBUG ] [ infer:131 ] Op: ConcatV2
[ 2021-02-07 20:13:15,866 ] [ DEBUG ] [ infer:132 ] Inputs:
[ 2021-02-07 20:13:15,868 ] [ DEBUG ] [ infer:32 ] input[0]: shape = [ 1 50625 1], value = <UNKNOWN>
[ 2021-02-07 20:13:15,869 ] [ DEBUG ] [ infer:32 ] input[1]: shape = [ 1 12996 1], value = <UNKNOWN>
[ 2021-02-07 20:13:15,870 ] [ DEBUG ] [ infer:32 ] input[2]: shape = [ 1 3249 1], value = <UNKNOWN>
[ 2021-02-07 20:13:15,872 ] [ DEBUG ] [ infer:32 ] input[3]: shape = [ 1 900 1], value = <UNKNOWN>
[ 2021-02-07 20:13:15,873 ] [ DEBUG ] [ infer:32 ] input[4]: shape = [ 1 225 1], value = <UNKNOWN>
[ 2021-02-07 20:13:15,874 ] [ DEBUG ] [ infer:32 ] input[5]: shape = [], value = 1
[ 2021-02-07 20:13:15,900 ] [ DEBUG ] [ infer:145 ] Outputs:
[ 2021-02-07 20:13:15,943 ] [ DEBUG ] [ infer:32 ] output[0]: shape = [ 1 67995 1], value = <UNKNOWN>
[ 2021-02-07 20:13:15,965 ] [ DEBUG ] [ infer:129 ] --------------------
[ 2021-02-07 20:13:15,966 ] [ DEBUG ] [ infer:130 ] Partial infer for StatefulPartitionedCall/retinanet-bbox/filtered_detections/map/TensorArrayUnstack_1/TensorListFromTensor
[ 2021-02-07 20:13:15,967 ] [ DEBUG ] [ infer:131 ] Op: TensorListFromTensor
[ 2021-02-07 20:13:15,967 ] [ DEBUG ] [ infer:132 ] Inputs:
[ 2021-02-07 20:13:15,970 ] [ DEBUG ] [ infer:32 ] input[0]: shape = [ 1 67995 1], value = <UNKNOWN>
[ 2021-02-07 20:13:15,971 ] [ DEBUG ] [ infer:32 ] input[1]: shape = [2], value = [-1 1]
[ INFO ] Called "tf_native_tf_node_infer" for node "StatefulPartitionedCall/retinanet-bbox/filtered_detections/map/TensorArrayUnstack_1/TensorListFromTensor"
[ 2021-02-07 20:13:15,979 ] [ DEBUG ] [ tf:228 ] Added placeholder with name 'StatefulPartitionedCall/retinanet-bbox/classification/concat_port_0_ie_placeholder'
[ 2021-02-07 20:13:15,993 ] [ DEBUG ] [ tf:228 ] Added placeholder with name 'StatefulPartitionedCall/retinanet-bbox/filtered_detections/map/TensorArrayUnstack_1/TensorListFromTensor/element_shape_port_0_ie_placeholder'
[ 2021-02-07 20:13:15,996 ] [ DEBUG ] [ tf:241 ] update_input_in_pbs: replace input 'StatefulPartitionedCall/retinanet-bbox/classification/concat' with input 'StatefulPartitionedCall/retinanet-bbox/classification/concat_port_0_ie_placeholder'
[ 2021-02-07 20:13:15,996 ] [ DEBUG ] [ tf:249 ] Replacing input '0' of the node 'StatefulPartitionedCall/retinanet-bbox/filtered_detections/map/TensorArrayUnstack_1/TensorListFromTensor' with placeholder 'StatefulPartitionedCall/retinanet-bbox/classification/concat_port_0_ie_placeholder'
[ 2021-02-07 20:13:15,999 ] [ DEBUG ] [ tf:241 ] update_input_in_pbs: replace input 'StatefulPartitionedCall/retinanet-bbox/filtered_detections/map/TensorArrayUnstack_1/TensorListFromTensor/element_shape' with input 'StatefulPartitionedCall/retinanet-bbox/filtered_detections/map/TensorArrayUnstack_1/TensorListFromTensor/element_shape_port_0_ie_placeholder'
[ 2021-02-07 20:13:16,002 ] [ DEBUG ] [ tf:249 ] Replacing input '1' of the node 'StatefulPartitionedCall/retinanet-bbox/filtered_detections/map/TensorArrayUnstack_1/TensorListFromTensor' with placeholder 'StatefulPartitionedCall/retinanet-bbox/filtered_detections/map/TensorArrayUnstack_1/TensorListFromTensor/element_shape_port_0_ie_placeholder'
2021-02-07 20:13:16.010402: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:02:00.0 name: Quadro RTX 3000 with Max-Q Design computeCapability: 7.5
coreClock: 1.305GHz coreCount: 30 deviceMemorySize: 6.00GiB deviceMemoryBandwidth: 268.26GiB/s
2021-02-07 20:13:16.074676: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2021-02-07 20:13:16.110707: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2021-02-07 20:13:16.116055: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2021-02-07 20:13:16.119597: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll
2021-02-07 20:13:16.122923: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll
2021-02-07 20:13:16.145155: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll
2021-02-07 20:13:16.148907: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll
2021-02-07 20:13:16.152697: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2021-02-07 20:13:16.156068: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2021-02-07 20:13:16.175746: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-02-07 20:13:16.182431: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267] 0
2021-02-07 20:13:16.186827: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0: N
2021-02-07 20:13:16.189289: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4594 MB memory) -> physical GPU (device: 0, name: Quadro RTX 3000 with Max-Q Design, pci bus id: 0000:02:00.0, compute capability: 7.5)
2021-02-07 20:13:16.312669: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-02-07 20:13:16.328833: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:196] None of the MLIR optimization passes are enabled (registered 0 passes)
[ ERROR ] Cannot infer shapes or values for node "StatefulPartitionedCall/retinanet-bbox/filtered_detections/map/TensorArrayUnstack_1/TensorListFromTensor".
[ ERROR ] Tensorflow type 21 not convertible to numpy dtype.
[ ERROR ]
[ ERROR ] It can happen due to bug in custom shape infer function <function tf_native_tf_node_infer at 0x000001DEA92CC9D8>.
[ ERROR ] Or because the node inputs have incorrect values/shapes.
[ ERROR ] Or because input shapes are incorrect (embedded to the model or passed via --input_shape).
[ 2021-02-07 20:13:16,408 ] [ DEBUG ] [ infer:197 ] Node "StatefulPartitionedCall/retinanet-bbox/filtered_detections/map/TensorArrayUnstack_1/TensorListFromTensor" attributes: {'pb': name: "StatefulPartitionedCall/retinanet-bbox/filtered_detections/map/TensorArrayUnstack_1/TensorListFromTensor"
op: "TensorListFromTensor"
input: "StatefulPartitionedCall/retinanet-bbox/classification/concat_port_0_ie_placeholder"
input: "StatefulPartitionedCall/retinanet-bbox/filtered_detections/map/TensorArrayUnstack_1/TensorListFromTensor/element_shape_port_0_ie_placeholder"
attr {
key: "_output_shapes"
value {
list {
shape {
}
}
}
}
attr {
key: "element_dtype"
value {
type: DT_FLOAT
}
}
attr {
key: "shape_type"
value {
type: DT_INT32
}
}
, '_in_ports': {0: {'control_flow': False}, 1: {'control_flow': False}}, '_out_ports': {0: {'control_flow': False}}, 'kind': 'op', 'name': 'StatefulPartitionedCall/retinanet-bbox/filtered_detections/map/TensorArrayUnstack_1/TensorListFromTensor', 'op': 'TensorListFromTensor', 'infer': <function tf_native_tf_node_infer at 0x000001DEA92CC9D8>, 'is_output_reachable': True, 'is_undead': False, 'is_const_producer': False, 'is_partial_inferred': False}
[ ERROR ] Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.PartialInfer.PartialInfer'>): Stopped shape/value propagation at "StatefulPartitionedCall/retinanet-bbox/filtered_detections/map/TensorArrayUnstack_1/TensorListFromTensor" node.
For more information please refer to Model Optimizer FAQ, question #38. (https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html?question=38#question-38)
[ 2021-02-07 20:13:16,832 ] [ DEBUG ] [ main:310 ] Traceback (most recent call last):
File "C:\Users\ashwinva\Miniconda3\envs\temp\lib\site-packages\tensorflow\python\client\session.py", line 1375, in _do_call
return fn(*args)
File "C:\Users\ashwinva\Miniconda3\envs\temp\lib\site-packages\tensorflow\python\client\session.py", line 1360, in _run_fn
target_list, run_metadata)
File "C:\Users\ashwinva\Miniconda3\envs\temp\lib\site-packages\tensorflow\python\client\session.py", line 1453, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InternalError: Tensorflow type 21 not convertible to numpy dtype.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Program Files (x86)\intel\openvino_2021\deployment_tools\model_optimizer\mo\middle\passes\infer.py", line 135, in partial_infer
node.infer(node)
File "C:\Program Files (x86)\intel\openvino_2021\deployment_tools\model_optimizer\mo\front\tf\partial_infer\tf.py", line 66, in tf_native_tf_node_infer
tf_subgraph_infer(tmp_node)
File "C:\Program Files (x86)\intel\openvino_2021\deployment_tools\model_optimizer\mo\front\tf\partial_infer\tf.py", line 141, in tf_subgraph_infer
all_constants, output_tensors = get_subgraph_output_tensors(node)
File "C:\Program Files (x86)\intel\openvino_2021\deployment_tools\model_optimizer\mo\front\tf\partial_infer\tf.py", line 126, in get_subgraph_output_tensors
result_tensor = sess.run(graph.get_tensor_by_name(out_tensor_name), feed_dict=feed_dict)
File "C:\Users\ashwinva\Miniconda3\envs\temp\lib\site-packages\tensorflow\python\client\session.py", line 968, in run
run_metadata_ptr)
File "C:\Users\ashwinva\Miniconda3\envs\temp\lib\site-packages\tensorflow\python\client\session.py", line 1191, in _run
feed_dict_tensor, options, run_metadata)
File "C:\Users\ashwinva\Miniconda3\envs\temp\lib\site-packages\tensorflow\python\client\session.py", line 1369, in _do_run
run_metadata)
File "C:\Users\ashwinva\Miniconda3\envs\temp\lib\site-packages\tensorflow\python\client\session.py", line 1394, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InternalError: Tensorflow type 21 not convertible to numpy dtype.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "C:\Program Files (x86)\intel\openvino_2021\deployment_tools\model_optimizer\mo\utils\class_registration.py", line 286, in apply_transform
replacer.find_and_replace_pattern(graph)
File "C:\Program Files (x86)\intel\openvino_2021\deployment_tools\model_optimizer\extensions\middle\PartialInfer.py", line 33, in find_and_replace_pattern
partial_infer(graph)
File "C:\Program Files (x86)\intel\openvino_2021\deployment_tools\model_optimizer\mo\middle\passes\infer.py", line 199, in partial_infer
refer_to_faq_msg(38)) from err
mo.utils.error.Error: Stopped shape/value propagation at "StatefulPartitionedCall/retinanet-bbox/filtered_detections/map/TensorArrayUnstack_1/TensorListFromTensor" node.
For more information please refer to Model Optimizer FAQ, question #38. (https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html?question=38#question-38)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "C:\Program Files (x86)\intel\openvino_2021\deployment_tools\model_optimizer\mo\main.py", line 297, in main
ret_code = driver(argv)
File "C:\Program Files (x86)\intel\openvino_2021\deployment_tools\model_optimizer\mo\main.py", line 264, in driver
ret_res = emit_ir(prepare_ir(argv), argv)
File "C:\Program Files (x86)\intel\openvino_2021\deployment_tools\model_optimizer\mo\main.py", line 233, in prepare_ir
graph = unified_pipeline(argv)
File "C:\Program Files (x86)\intel\openvino_2021\deployment_tools\model_optimizer\mo\pipeline\unified.py", line 29, in unified_pipeline
class_registration.ClassType.BACK_REPLACER
File "C:\Program Files (x86)\intel\openvino_2021\deployment_tools\model_optimizer\mo\utils\class_registration.py", line 338, in apply_replacements
apply_replacements_list(graph, replacers_order)
File "C:\Program Files (x86)\intel\openvino_2021\deployment_tools\model_optimizer\mo\utils\class_registration.py", line 328, in apply_replacements_list
num_transforms=len(replacers_order))
File "C:\Program Files (x86)\intel\openvino_2021\deployment_tools\model_optimizer\mo\utils\logger.py", line 124, in wrapper
function(*args, **kwargs)
File "C:\Program Files (x86)\intel\openvino_2021\deployment_tools\model_optimizer\mo\utils\class_registration.py", line 308, in apply_transform
)) from err
mo.utils.error.Error: Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.PartialInfer.PartialInfer'>): Stopped shape/value propagation at "StatefulPartitionedCall/retinanet-bbox/filtered_detections/map/TensorArrayUnstack_1/TensorListFromTensor" node.
For more information please refer to Model Optimizer FAQ, question #38. (https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html?question=38#question-38)

 

 

0 Kudos
1 Solution
Zulkifli_Intel
Moderator
878 Views

Hi Ashwin,

Thank you for contacting us.

OpenVINO™ support for TensorFlow 2 models is in preview (aka Beta), which means limited and not of production quality yet. This information is available at the following page:

https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow.html#keras_h5

 

I suggest you use Retinanet-tf, which is a public model available with OpenVINO from Open Model Zoo. You can download it using Model Downloader.

The steps to convert the model to TensorFlow is given at the following link:

https://docs.openvinotoolkit.org/latest/omz_models_public_retinanet_tf_retinanet_tf.html#steps_to_reproduce_keras_to_tensorflow_conversion

 

After that, convert to IR format using Model Optimizer. The Model Optimizer arguments are given here:

https://github.com/openvinotoolkit/open_model_zoo/blob/master/models/public/retinanet-tf/model.yml#L25

 

Regards,

Zulkifli


View solution in original post

0 Kudos
2 Replies
Zulkifli_Intel
Moderator
879 Views

Hi Ashwin,

Thank you for contacting us.

OpenVINO™ support for TensorFlow 2 models is in preview (aka Beta), which means limited and not of production quality yet. This information is available at the following page:

https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow.html#keras_h5

 

I suggest you use Retinanet-tf, which is a public model available with OpenVINO from Open Model Zoo. You can download it using Model Downloader.

The steps to convert the model to TensorFlow is given at the following link:

https://docs.openvinotoolkit.org/latest/omz_models_public_retinanet_tf_retinanet_tf.html#steps_to_reproduce_keras_to_tensorflow_conversion

 

After that, convert to IR format using Model Optimizer. The Model Optimizer arguments are given here:

https://github.com/openvinotoolkit/open_model_zoo/blob/master/models/public/retinanet-tf/model.yml#L25

 

Regards,

Zulkifli


0 Kudos
Zulkifli_Intel
Moderator
832 Views

Hi Ashwin,

 

This thread will no longer be monitored since we have provided a solution. If you need any additional information from Intel, please submit a new question.

 

Regards

Zulkifli

 

0 Kudos
Reply