Exception during running replacer ONNXMaskRCNNReplacement in conversion of MaskRCNN from ONNX to IR

[Goyal__Pranav]

Hi,

In an attempt to convert a MaskRCNN model from Pytorch to ONNX to IR, it's failing in the conversion from ONNX to IR (ONNXMaskRCNNReplacement is attempting to access a node that's not in the graph). I am using OpenVINO version 2020.1

CONVERSION DETAILS -

I used ONNX 1.6 and opset version 11 for the conversion to ONNX. 

For converting to IR, this was the input command: 

sudo python3 $model_optimizer/mo.py --input_model mask_rcnn_R_50_FPN_1x.onnx --input_shape "[1,3,960,1280]"  --transformations_config $model_optimizer/extensions/front/onnx/mask_rcnn.json --log_level DEBUG --input "0:2"

Error log:

[ ERROR ]  -------------------------------------------------
[ ERROR ]  ----------------- INTERNAL ERROR ----------------
[ ERROR ]  Unexpected exception happened.
[ ERROR ]  Please contact Model Optimizer developers and forward the following information:
[ ERROR ]  Exception occurred during running replacer "ONNXMaskRCNNReplacement (<class 'extensions.front.onnx.mask_rcnn_conversion.ObjectDetectionAPIOutputReplacement'>)": Attempt to access node 6795 that not in graph
[ ERROR ]  Traceback (most recent call last):
  File "/opt/intel/openvino_2020.1.023/deployment_tools/model_optimizer/mo/utils/class_registration.py", line 287, in apply_transform
    for_graph_and_each_sub_graph_recursively(graph, replacer.find_and_replace_pattern)
  File "/opt/intel/openvino_2020.1.023/deployment_tools/model_optimizer/mo/middle/pattern_match.py", line 58, in for_graph_and_each_sub_graph_recursively
    func(graph)
  File "/opt/intel/openvino_2020.1.023/deployment_tools/model_optimizer/mo/front/tf/replacement.py", line 48, in find_and_replace_pattern
    self.transform_graph(graph, desc._replacement_desc['custom_attributes'])
  File "/opt/intel/openvino_2020.1.023/deployment_tools/model_optimizer/extensions/front/onnx/mask_rcnn_conversion.py", line 49, in transform_graph
    insert_ExperimentalDetectronROIFeatureExtractor1(graph)
  File "/opt/intel/openvino_2020.1.023/deployment_tools/model_optimizer/extensions/front/onnx/mask_rcnn_conversion.py", line 87, in insert_ExperimentalDetectronROIFeatureExtractor1
    old_output_node = Node(graph, '6795')
  File "/opt/intel/openvino_2020.1.023/deployment_tools/model_optimizer/mo/graph/graph.py", line 38, in __init__
    assert node in graph, "Attempt to access node {} that not in graph".format(node)
AssertionError: Attempt to access node 6795 that not in graph

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/intel/openvino_2020.1.023/deployment_tools/model_optimizer/mo/main.py", line 314, in main
    return driver(argv)
  File "/opt/intel/openvino_2020.1.023/deployment_tools/model_optimizer/mo/main.py", line 281, in driver
    ret_res = emit_ir(prepare_ir(argv), argv)
  File "/opt/intel/openvino_2020.1.023/deployment_tools/model_optimizer/mo/main.py", line 246, in prepare_ir
    graph = mo_onnx.driver(argv)
  File "/opt/intel/openvino_2020.1.023/deployment_tools/model_optimizer/mo/pipeline/onnx.py", line 72, in driver
    class_registration.ClassType.BACK_REPLACER
  File "/opt/intel/openvino_2020.1.023/deployment_tools/model_optimizer/mo/utils/class_registration.py", line 324, in apply_replacements
    num_transforms=len(replacers_order))
  File "/opt/intel/openvino_2020.1.023/deployment_tools/model_optimizer/mo/utils/logger.py", line 124, in wrapper
    function(*args, **kwargs)
  File "/opt/intel/openvino_2020.1.023/deployment_tools/model_optimizer/mo/utils/class_registration.py", line 309, in apply_transform
    )) from err
Exception: Exception occurred during running replacer "ONNXMaskRCNNReplacement (<class 'extensions.front.onnx.mask_rcnn_conversion.ObjectDetectionAPIOutputReplacement'>)": Attempt to access node 6795 that not in graph

Please advise on how to solve this problem.

Thank you

[SIRIGIRI_V_Intel]

Hi Pranav,

Can you confirm that you are using the Supported Pytorch to ONNX topologies.

A supported Mask R-CNN ONNX model conversion to IR is provided by the OpenVINO. Can you please have look.

Regards,

Ram prasad

[Goyal__Pranav]

Thank you for your response.

I am using an ONNX compatible model of facebookresearch/maskrcnn-benchmark model as found on https://github.com/BowenBao/maskrcnn-benchmark/tree/onnx_stage, which gave a smooth conversion to ONNX. However, the Pytorch model clearly doesn't belong to Supported Pytorch to ONNX topologies(that you shared above). Does that mean it's not possible to convert it to IR ?

I did try using the supported Mask R-CNN ONNX model as provided by OpenVino and was able to successfully convert it. Can it be told how this model was converted from Pytorch ?

For clarification, my goal is to convert another model Pyramid Mask Text Detector(PMTD), that is closely derived from Mask R-CNN. So the course of action I intended to follow were:

1) Convert Mask R-CNN from Pytorch to ONNX to IR.

2) Once I have the IR compatible model of Mask R-CNN in Pytorch, I will make the required changes in it to obtain PMTD in Pytorch.

3) Convert PMTD from Pytorch to ONNX to IR. 

But since the problem lies in the conversion of Mask R-CNN from ONNX to IR, I am unable to proceed. Could you please suggest what I should do? Looking forward to an early response. 

Regards