Failed to convert Tensorflow pb model to .xml/.bin with Model Optimizer

Chen__Brandon · ‎04-08-2020

Tested with Openvino 2019_R3.1. It failed when I tried to convert a tf pb file to .xml/.bin with model optimizer and got bellow errors:

(vino) PS C:\Program Files (x86)\IntelSWTools\openvino_2019.3.379\deployment_tools\model_optimizer> python.exe .\mo_tf.py --input_model "C:\Users\chen\Workspace\VINO\ma
tting\mmnet1.4.pb" --input_shape [1,256,256,3] --output_dir "C:\Users\chen\Workspace\VINO\models\mobilenet"
Model Optimizer arguments:
Common parameters:
        - Path to the Input Model:      C:\Users\chen\Workspace\VINO\matting\mmnet1.4.pb
        - Path for generated IR:        C:\Users\chen\Workspace\VINO\models\mobilenet
        - IR output name:       mmnet1.4
        - Log level:    ERROR
        - Batch:        Not specified, inherited from the model
        - Input layers:         Not specified, inherited from the model
        - Output layers:        Not specified, inherited from the model
        - Input shapes:         [1,256,256,3]
        - Mean values:  Not specified
        - Scale values:         Not specified
        - Scale factor:         Not specified
        - Precision of IR:      FP32
        - Enable fusing:        True
        - Enable grouped convolutions fusing:   True
        - Move mean values to preprocess section:       False
        - Reverse input channels:       False
TensorFlow specific parameters:
        - Input model in text protobuf format:  False
        - Path to model dump for TensorBoard:   None
        - List of shared libraries with TensorFlow custom layers implementation:        None
        - Update the configuration file with input/output node names:   None
        - Use configuration file used to generate the model with Object Detection API:  None
        - Operations to offload:        None
        - Patterns to offload:  None
        - Use the config file:  None
Model Optimizer version:        2019.3.0-408-gac8584cb7
[ ERROR ]  -------------------------------------------------
[ ERROR ]  ----------------- INTERNAL ERROR ----------------
[ ERROR ]  Unexpected exception happened.
[ ERROR ]  Please contact Model Optimizer developers and forward the following information:
[ ERROR ]  Exception occurred during running replacer "REPLACEMENT_ID (<class 'extensions.back.ReduceToPooling.ReduceReplacer'>)": The node "MMNet/enc_block4/branch1/depthwise_conv_dilation/BatchNorm/moments/variance" is a Reduce operation for batch dimension which is not supported
[ ERROR ]  Traceback (most recent call last):
  File "C:\Program Files (x86)\IntelSWTools\openvino_2019.3.379\deployment_tools\model_optimizer\mo\utils\class_registration.py", line 273, in apply_replacements
    for_graph_and_each_sub_graph_recursively(graph, replacer.find_and_replace_pattern)
  File "C:\Program Files (x86)\IntelSWTools\openvino_2019.3.379\deployment_tools\model_optimizer\mo\middle\pattern_match.py", line 58, in for_graph_and_each_sub_graph_recursively
    func(graph)
  File "C:\Program Files (x86)\IntelSWTools\openvino_2019.3.379\deployment_tools\model_optimizer\mo\utils\replacement_pattern.py", line 29, in find_and_replace_pattern
    apply_pattern(graph, **self.pattern(), action=self.replace_pattern)  # pylint: disable=no-member
  File "C:\Program Files (x86)\IntelSWTools\openvino_2019.3.379\deployment_tools\model_optimizer\mo\middle\pattern_match.py", line 95, in apply_pattern
    action(graph, match)
  File "C:\Program Files (x86)\IntelSWTools\openvino_2019.3.379\deployment_tools\model_optimizer\extensions\back\ReduceToPooling.py", line 82, in replace_pattern
    node.name)
AssertionError: The node "MMNet/enc_block4/branch1/depthwise_conv_dilation/BatchNorm/moments/variance" is a Reduce operation for batch dimension which is not supported

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Program Files (x86)\IntelSWTools\openvino_2019.3.379\deployment_tools\model_optimizer\mo\main.py", line 298, in main
    return driver(argv)
  File "C:\Program Files (x86)\IntelSWTools\openvino_2019.3.379\deployment_tools\model_optimizer\mo\main.py", line 247, in driver
    is_binary=not argv.input_model_is_text)
  File "C:\Program Files (x86)\IntelSWTools\openvino_2019.3.379\deployment_tools\model_optimizer\mo\pipeline\tf.py", line 227, in tf2nx
    class_registration.apply_replacements(graph, class_registration.ClassType.BACK_REPLACER)
  File "C:\Program Files (x86)\IntelSWTools\openvino_2019.3.379\deployment_tools\model_optimizer\mo\utils\class_registration.py", line 299, in apply_replacements
    )) from err
Exception: Exception occurred during running replacer "REPLACEMENT_ID (<class 'extensions.back.ReduceToPooling.ReduceReplacer'>)": The node "MMNet/enc_block4/branch1/depthwise_conv_dilation/BatchNorm/moments/variance" is a Reduce operation for batch dimension which is not supported

[ ERROR ]  ---------------- END OF BUG REPORT --------------
[ ERROR ]  -------------------------------------------------

How to fix such errors?

Thanks.

SuryaPSC_Intel · ‎04-09-2020

Hi Brandon,

Can you please share what is the topology of your model?

Best Regards,

Surya

Chen__Brandon · ‎04-10-2020

Hi Surya,

Thanks for your reply.

After replacing BN layer with FusedBatchNorm in the model, previous errors are gone but new errors occur that spacetobatchnd/batchtospacend not supported:

Model Optimizer arguments:
Common parameters:
        - Path to the Input Model:      C:\Users\branchen\Workspace\Realtime_Person_Detection\matting\mmnet2.0.pb
        - Path for generated IR:        C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\model_optimizer\.
        - IR output name:       mmnet2.0
        - Log level:    ERROR
        - Batch:        Not specified, inherited from the model
        - Input layers:         Not specified, inherited from the model
        - Output layers:        Not specified, inherited from the model
        - Input shapes:         [1,256,256,3]
        - Mean values:  Not specified
        - Scale values:         Not specified
        - Scale factor:         Not specified
        - Precision of IR:      FP32
        - Enable fusing:        True
        - Enable grouped convolutions fusing:   True
        - Move mean values to preprocess section:       False
        - Reverse input channels:       False
TensorFlow specific parameters:
        - Input model in text protobuf format:  False
        - Path to model dump for TensorBoard:   None
        - List of shared libraries with TensorFlow custom layers implementation:        None
        - Update the configuration file with input/output node names:   None
        - Use configuration file used to generate the model with Object Detection API:  None
        - Operations to offload:        None
        - Patterns to offload:  None
        - Use the config file:  None
Model Optimizer version:        2019.3.0-408-gac8584cb7
[ ERROR ]  List of operations that cannot be converted to Inference Engine IR:
[ ERROR ]      SpaceToBatchND (26)
[ ERROR ]          MMNet/enc_block0/branch1/depthwise_conv_dilation/SpaceToBatchND
[ ERROR ]          MMNet/enc_block0/branch2/depthwise_conv_dilation/SpaceToBatchND
[ ERROR ]          MMNet/enc_block0/branch3/depthwise_conv_dilation/SpaceToBatchND
[ ERROR ]          MMNet/enc_block1/branch1/depthwise_conv_dilation/SpaceToBatchND
[ ERROR ]          MMNet/enc_block1/branch2/depthwise_conv_dilation/SpaceToBatchND
[ ERROR ]          MMNet/enc_block1/branch3/depthwise_conv_dilation/SpaceToBatchND
[ ERROR ]          MMNet/enc_block2/branch1/depthwise_conv_dilation/SpaceToBatchND
[ ERROR ]          MMNet/enc_block2/branch2/depthwise_conv_dilation/SpaceToBatchND
[ ERROR ]          MMNet/enc_block2/branch3/depthwise_conv_dilation/SpaceToBatchND
[ ERROR ]          MMNet/enc_block3/branch1/depthwise_conv_dilation/SpaceToBatchND
[ ERROR ]          MMNet/enc_block3/branch2/depthwise_conv_dilation/SpaceToBatchND
[ ERROR ]          MMNet/enc_block3/branch3/depthwise_conv_dilation/SpaceToBatchND
[ ERROR ]          MMNet/enc_block4/branch1/depthwise_conv_dilation/SpaceToBatchND
[ ERROR ]          MMNet/enc_block4/branch2/depthwise_conv_dilation/SpaceToBatchND
[ ERROR ]          MMNet/enc_block5/branch1/depthwise_conv_dilation/SpaceToBatchND
[ ERROR ]          MMNet/enc_block5/branch2/depthwise_conv_dilation/SpaceToBatchND
[ ERROR ]          MMNet/enc_block6/branch1/depthwise_conv_dilation/SpaceToBatchND
[ ERROR ]          MMNet/enc_block6/branch2/depthwise_conv_dilation/SpaceToBatchND
[ ERROR ]          MMNet/enc_block7/branch1/depthwise_conv_dilation/SpaceToBatchND
[ ERROR ]          MMNet/enc_block7/branch2/depthwise_conv_dilation/SpaceToBatchND
[ ERROR ]          MMNet/enc_block8/branch1/depthwise_conv_dilation/SpaceToBatchND
[ ERROR ]          MMNet/enc_block9/branch1/depthwise_conv_dilation/SpaceToBatchND
[ ERROR ]          MMNet/dec_block2/branch1/depthwise_conv_dilation/SpaceToBatchND
[ ERROR ]          MMNet/dec_block2/branch2/depthwise_conv_dilation/SpaceToBatchND
[ ERROR ]          MMNet/dec_block3/branch1/depthwise_conv_dilation/SpaceToBatchND
[ ERROR ]          MMNet/dec_block3/branch2/depthwise_conv_dilation/SpaceToBatchND
[ ERROR ]      BatchToSpaceND (26)
[ ERROR ]          MMNet/enc_block0/branch1/depthwise_conv_dilation/BatchToSpaceND
[ ERROR ]          MMNet/enc_block0/branch2/depthwise_conv_dilation/BatchToSpaceND
[ ERROR ]          MMNet/enc_block0/branch3/depthwise_conv_dilation/BatchToSpaceND
[ ERROR ]          MMNet/enc_block1/branch1/depthwise_conv_dilation/BatchToSpaceND
[ ERROR ]          MMNet/enc_block1/branch2/depthwise_conv_dilation/BatchToSpaceND
[ ERROR ]          MMNet/enc_block1/branch3/depthwise_conv_dilation/BatchToSpaceND
[ ERROR ]          MMNet/enc_block2/branch1/depthwise_conv_dilation/BatchToSpaceND
[ ERROR ]          MMNet/enc_block2/branch2/depthwise_conv_dilation/BatchToSpaceND
[ ERROR ]          MMNet/enc_block2/branch3/depthwise_conv_dilation/BatchToSpaceND
[ ERROR ]          MMNet/enc_block3/branch1/depthwise_conv_dilation/BatchToSpaceND
[ ERROR ]          MMNet/enc_block3/branch2/depthwise_conv_dilation/BatchToSpaceND
[ ERROR ]          MMNet/enc_block3/branch3/depthwise_conv_dilation/BatchToSpaceND
[ ERROR ]          MMNet/enc_block4/branch1/depthwise_conv_dilation/BatchToSpaceND
[ ERROR ]          MMNet/enc_block4/branch2/depthwise_conv_dilation/BatchToSpaceND
[ ERROR ]          MMNet/enc_block5/branch1/depthwise_conv_dilation/BatchToSpaceND
[ ERROR ]          MMNet/enc_block5/branch2/depthwise_conv_dilation/BatchToSpaceND
[ ERROR ]          MMNet/enc_block6/branch1/depthwise_conv_dilation/BatchToSpaceND
[ ERROR ]          MMNet/enc_block6/branch2/depthwise_conv_dilation/BatchToSpaceND
[ ERROR ]          MMNet/enc_block7/branch1/depthwise_conv_dilation/BatchToSpaceND
[ ERROR ]          MMNet/enc_block7/branch2/depthwise_conv_dilation/BatchToSpaceND
[ ERROR ]          MMNet/enc_block8/branch1/depthwise_conv_dilation/BatchToSpaceND
[ ERROR ]          MMNet/enc_block9/branch1/depthwise_conv_dilation/BatchToSpaceND
[ ERROR ]          MMNet/dec_block2/branch1/depthwise_conv_dilation/BatchToSpaceND
[ ERROR ]          MMNet/dec_block2/branch2/depthwise_conv_dilation/BatchToSpaceND
[ ERROR ]          MMNet/dec_block3/branch1/depthwise_conv_dilation/BatchToSpaceND
[ ERROR ]          MMNet/dec_block3/branch2/depthwise_conv_dilation/BatchToSpaceND
[ ERROR ]  Part of the nodes was not converted to IR. Stopped.
 For more information please refer to Model Optimizer FAQ (https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html), question #24.

Attached it the model topology.

Thanks & Regards,

Brandon

SuryaPSC_Intel · ‎04-10-2020

Hi Brandon,

Please confirm if it is one of the supported topologies.
If it is a custom model kindly ensure, all the layers are supported.

As per the error, some of the layers are not supported by the Inference Engine and cannot be translated to an Intermediate Representation.

You can extend the Model Optimizer by allowing generation of new types of layers and implement these layers in the dedicated Inference Engine plugins. For more information, refer to Extending the Model Optimizer with New Primitives page and Inference Engine Kernels Extensibility.

Best Regards,

Surya