Error while converting frozen graph to IR

[SSong16]

Hello everyone, I am working on the Movidius Neural Compute Stick 2 with OpenVino Toolkit. The model I am using is Faster R-CNN Inception V2 COCO. I have retrained the model to detect 4 classes of my own and exported and freeze the inference graph. However, when I'm trying to convert the frozen inference graph to IR in OpenVino, I have encountered the following error: 

Model Optimizer arguments:
Common parameters:
    - Path to the Input Model:     /home/vudthi/inference_graph/frozen_inference_graph.pb
    - Path for generated IR:     /opt/intel/computer_vision_sdk_2018.5.455/deployment_tools/model_optimizer/.
    - IR output name:     frozen_inference_graph
    - Log level:     ERROR
    - Batch:     Not specified, inherited from the model
    - Input layers:     image_tensor
    - Output layers:     Not specified, inherited from the model
    - Input shapes:     [1,224,224,3]
    - Mean values:     Not specified
    - Scale values:     Not specified
    - Scale factor:     Not specified
    - Precision of IR:     FP32
    - Enable fusing:     True
    - Enable grouped convolutions fusing:     True
    - Move mean values to preprocess section:     False
    - Reverse input channels:     False
TensorFlow specific parameters:
    - Input model in text protobuf format:     False
    - Offload unsupported operations:     False
    - Path to model dump for TensorBoard:     None
    - List of shared libraries with TensorFlow custom layers implementation:     None
    - Update the configuration file with input/output node names:     None
    - Use configuration file used to generate the model with Object Detection API:     None
    - Operations to offload:     None
    - Patterns to offload:     None
    - Use the config file:     None
Model Optimizer version:     1.5.12.49d067a0
/opt/intel/computer_vision_sdk_2018.5.455/deployment_tools/model_optimizer/mo/ops/slice.py:111: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
  value = value[slice_idx]
[ ERROR ]  Cannot infer shapes or values for node "Preprocessor/map/while/ResizeToRange/Round_3".
[ ERROR ]  Input 0 of node Preprocessor/map/while/ResizeToRange/Round_3 was passed double from Preprocessor/map/while/ResizeToRange/mul_3_port_0_ie_placeholder:0 incompatible with expected float.
[ ERROR ]  
[ ERROR ]  It can happen due to bug in custom shape infer function <function tf_native_tf_node_infer at 0x7f5f4e74b268>.
[ ERROR ]  Or because the node inputs have incorrect values/shapes.
[ ERROR ]  Or because input shapes are incorrect (embedded to the model or passed via --input_shape).
[ ERROR ]  Run Model Optimizer with --log_level=DEBUG for more information.
[ ERROR ]  Stopped shape/value propagation at "Preprocessor/map/while/ResizeToRange/Round_3" node. 
 

My script looks something like this: sudo python3 mo_tf.py --input_model <dir>/frozen_inference_graph.pb --input_shape [1,224,224,3] --input image_tensor --data_type FP32

I've seen similar problem on the forum and I tried their solution also, but it didn't work for me. Hoping to find the solution! Thanks

[Shubha_R_Intel]

Dear Sokha Vudthi, please use --log_level DEBUG in your Model Optimizer command and paste the ERROR part here.  I can help you if I see the detailed error.

Thanks !

Shubha

[SSong16]

Dear Shubha R., I've added --log_level=DEBUG and the following are the ERROR part:

[ ERROR ]  Cannot infer shapes or values for node "Preprocessor/map/while/ResizeToRange/Round_1".
[ ERROR ]  Input 0 of node Preprocessor/map/while/ResizeToRange/Round_1 was passed double from Preprocessor/map/while/ResizeToRange/mul_1_port_0_ie_placeholder:0 incompatible with expected float.
[ ERROR ]  
[ ERROR ]  It can happen due to bug in custom shape infer function <function tf_native_tf_node_infer at 0x7f166918e268>.
[ ERROR ]  Or because the node inputs have incorrect values/shapes.
[ ERROR ]  Or because input shapes are incorrect (embedded to the model or passed via --input_shape).
[ 2019-03-25 13:47:20,750 ] [ DEBUG ] [ infer:215 ]  Node "Preprocessor/map/while/ResizeToRange/Round_1" attributes: {'precision': 'FP32', 'dim_attrs': ['axis', 'batch_dims', 'channel_dims', 'spatial_dims'], 'is_output_reachable': True, 'is_undead': False, 'IE': [('layer', [('id', <function update_ie_fields.<locals>.<lambda> at 0x7f1667290ea0>), 'name', 'precision', 'type'], [('data', ['auto_pad', 'epsilon', 'min', 'max', ('axis', <function update_ie_fields.<locals>.<lambda> at 0x7f1667290f28>), 'tiles', ('dim', <function update_ie_fields.<locals>.<lambda> at 0x7f1667291048>), 'num_axes', ('pool-method', 'pool_method'), 'group', ('rounding-type', 'rounding_type'), ('exclude-pad', 'exclude_pad'), 'operation', 'out-size', 'power', 'shift', 'alpha', 'beta', 'coords', 'classes', 'num', ('local-size', 'local_size'), 'region', 'knorm', 'num_classes', 'keep_top_k', 'variance_encoded_in_target', 'code_type', 'share_location', 'nms_threshold', 'confidence_threshold', 'background_label_id', 'top_k', 'eta', 'visualize', 'visualize_threshold', 'save_file', 'output_directory', 'output_name_prefix', 'output_format', 'label_map_file', 'name_size_file', 'num_test_image', 'prob', 'resize_mode', 'height', 'width', 'height_scale', 'width_scale', 'pad_mode', 'pad_value', 'interp_mode', 'img_size', 'img_h', 'img_w', 'step', 'step_h', 'step_w', ('offset', <function update_ie_fields.<locals>.<lambda> at 0x7f1667291158>), 'variance', 'flip', 'clip', ('min_size', <function update_ie_fields.<locals>.<lambda> at 0x7f16672910d0>), ('max_size', <function update_ie_fields.<locals>.<lambda> at 0x7f1667291268>), ('aspect_ratio', <function update_ie_fields.<locals>.<lambda> at 0x7f16672911e0>), 'decrease_label_id', 'normalized', 'scale_all_sizes', ('type', 'norm_type'), 'eps', 'across_spatial', 'channel_shared', 'negative_slope', 'engine', 'num_filter', ('type', 'sample_type'), ('order', <function update_ie_fields.<locals>.<lambda> at 0x7f1667291378>), 'pooled_h', 'pooled_w', 'spatial_scale', 'cls_threshold', 'max_num_proposals', 'iou_threshold', 'min_bbox_size', 'feat_stride', 'pre_nms_topn', 'post_nms_topn', ('type', <function update_ie_fields.<locals>.<lambda> at 0x7f16672912f0>), ('value', <function update_ie_fields.<locals>.<lambda> at 0x7f1667291400>), ('output', <function update_ie_fields.<locals>.<lambda> at 0x7f1667291488>), ('input_nodes_names', <function update_ie_fields.<locals>.<lambda> at 0x7f1667291510>), ('output_tensors_names', <function update_ie_fields.<locals>.<lambda> at 0x7f1667291598>), ('real_input_dims', <function update_ie_fields.<locals>.<lambda> at 0x7f1667291620>), ('protobuf', <function update_ie_fields.<locals>.<lambda> at 0x7f16672916a8>), {'custom_attributes': None}, ('strides', <function update_ie_fields.<locals>.<lambda> at 0x7f1667291730>), ('kernel', <function update_ie_fields.<locals>.<lambda> at 0x7f16672917b8>), ('dilations', <function update_ie_fields.<locals>.<lambda> at 0x7f1667291840>), ('pads_begin', <function update_ie_fields.<locals>.<lambda> at 0x7f16672918c8>), ('pads_end', <function update_ie_fields.<locals>.<lambda> at 0x7f1667291950>), ('scale', <function update_ie_fields.<locals>.<lambda> at 0x7f16672919d8>), 'crop_width', 'crop_height', 'write_augmented', 'max_multiplier', 'augment_during_test', 'recompute_mean', 'write_mean', 'mean_per_pixel', 'mode', 'bottomwidth', 'bottomheight', 'chromatic_eigvec', 'kernel_size', 'max_displacement', 'stride_1', 'stride_2', 'single_direction', 'do_abs', 'correlation_type', 'antialias', 'resample_type', 'factor', 'coeff', ('ratio', <function update_ie_fields.<locals>.<lambda> at 0x7f1667291a60>)], []), '@ports', '@consts'])], 'kind': 'op', 'is_const_producer': False, 'is_partial_inferred': False, 'infer': <function tf_native_tf_node_infer at 0x7f166918e268>, 'pb': name: "Preprocessor/map/while/ResizeToRange/Round_1"
op: "Round"
input: "Preprocessor/map/while/ResizeToRange/mul_1_port_0_ie_placeholder"
attr {
  key: "T"
  value {
    type: DT_FLOAT
  }
}
, 'shape_attrs': ['window', 'pad', 'shape', 'output_shape', 'stride'], 'name': 'Preprocessor/map/while/ResizeToRange/Round_1', 'op': 'Round'}
[ ERROR ]  Stopped shape/value propagation at "Preprocessor/map/while/ResizeToRange/Round_1" node. 
 For more information please refer to Model Optimizer FAQ (<INSTALL_DIR>/deployment_tools/documentation/docs/MO_FAQ.html), question #38. 
[ 2019-03-25 13:47:20,751 ] [ DEBUG ] [ main:331 ]  Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/importer.py", line 426, in import_graph_def
    graph._c_graph, serialized, options)  # pylint: disable=protected-access
tensorflow.python.framework.errors_impl.InvalidArgumentError: Input 0 of node Preprocessor/map/while/ResizeToRange/Round_1 was passed double from Preprocessor/map/while/ResizeToRange/mul_1_port_0_ie_placeholder:0 incompatible with expected float.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/intel/computer_vision_sdk_2018.5.455/deployment_tools/model_optimizer/mo/middle/passes/infer.py", line 153, in partial_infer
    node.infer(node)
  File "/opt/intel/computer_vision_sdk_2018.5.455/deployment_tools/model_optimizer/mo/front/tf/partial_infer/tf.py", line 60, in tf_native_tf_node_infer
    tf_subgraph_infer(tmp_node)
  File "/opt/intel/computer_vision_sdk_2018.5.455/deployment_tools/model_optimizer/mo/front/tf/partial_infer/tf.py", line 135, in tf_subgraph_infer
    all_constants, output_tensors = get_subgraph_output_tensors(node)
  File "/opt/intel/computer_vision_sdk_2018.5.455/deployment_tools/model_optimizer/mo/front/tf/partial_infer/tf.py", line 115, in get_subgraph_output_tensors
    tf.import_graph_def(graph_def, name='')
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/importer.py", line 430, in import_graph_def
    raise ValueError(str(e))
ValueError: Input 0 of node Preprocessor/map/while/ResizeToRange/Round_1 was passed double from Preprocessor/map/while/ResizeToRange/mul_1_port_0_ie_placeholder:0 incompatible with expected float.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/intel/computer_vision_sdk_2018.5.455/deployment_tools/model_optimizer/mo/main.py", line 325, in main
    return driver(argv)
  File "/opt/intel/computer_vision_sdk_2018.5.455/deployment_tools/model_optimizer/mo/main.py", line 267, in driver
    mean_scale_values=mean_scale)
  File "/opt/intel/computer_vision_sdk_2018.5.455/deployment_tools/model_optimizer/mo/pipeline/tf.py", line 256, in tf2nx
    partial_infer(graph)
  File "/opt/intel/computer_vision_sdk_2018.5.455/deployment_tools/model_optimizer/mo/middle/passes/infer.py", line 217, in partial_infer
    refer_to_faq_msg(38)) from err
mo.utils.error.Error: Stopped shape/value propagation at "Preprocessor/map/while/ResizeToRange/Round_1" node. 

Thanks for the help!

Sokha Vudthi

[Shubha_R_Intel]

Dear Sokha:

Please give me the exact link from where you downloaded Faster R-CNN Inception V2 COCO. 

We do support Faster R-CNN, please check this list for supported topologies. We supply the exact tar.gz which you should download. Did you try one of these ?

https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow.html

Shubha

[SSong16]

Dear Shubha,

I downloaded the pre-trained model from the TensorFlow model zoo. The following is the link to their github repository:

https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md

I haven't tried the ones from the openvinotoolkit website yet since I trained my model a few months back before I found out about Intel Movidius Neural Compute Stick.

I will download the exact tar.gz from openvino website, will try to convert it again and see how it goes.

Thanks!

Vudthi  

[SSong16]

Dear Shubha,

I've downloaded the model from openvino website and tried to convert it again but it still shows the same error.

I've also tried a few other topologies also but all of them seem to output the same error as the previous one.

Vudthi

[Mikhail_T_Intel]

Hi,

For proper conversion of Tensorflow Object Detection API models you usually have to specify two additional comand line options

  --tensorflow_use_custom_operations_config $MO_ROOT/extensions/front/tf/faster_rcnn_support.json for models frozen with TF version < 1.7; faster_rcnn_support_api_v1.7.json for TF version from 1.7 to 1.10 and faster_rcnn_support_api_v1.10.json for TF version starting from 1.10

--tensorflow_object_detection_api_pipeline_config pipeline.config The pipeline.config file is provided along with the models from Tensorflow detection model zoo.

Here is more detailed explanation about how MO works with Faster R-CNN TF models, and the paper about the general approach to the models created using the TensorFlow Object Detection API

[om77]

Just to add my 5 cents here...

frozen_inference_graph.pb should be obtained using tensorflow version <= 1.12 since the last tensorflow (e.g. 1.13.1) is not supported by openvino R5.

[SSong16]

Dear Mikhail, om77, and Shubha,

I have successfully converted the frozen inference graph into IR now after adding the custom operation config file.

Thank you very much for all the helps!

Vudthi

[Shubha_R_Intel]

Dear Sokha Vudthi,

congrats and thanks for using OpenVino !

Shubha