Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6392 Discussions

Convert Pytorch to ONNX then to Openvino IR

kenfack
Beginner
967 Views

I have trained a mask rcnn model using transfers learning in PyTorch on the custom dataset.

 

num_classes = 4
model = models.detection.maskrcnn_resnet50_fpn(pretrained=True)
in_features = model.roi_heads.box_predictor.cls_score.in_features
model.roi_heads.box_predictor = FastRCNNPredictor(in_features, num_classes)
in_features_mask = model.roi_heads.mask_predictor.conv5_mask.in_channels
hidden_layer = 256
model.roi_heads.mask_predictor = MaskRCNNPredictor(in_features_mask,hidden_layer,num_classes)

 

Now I need to covert the resulted model into ONNX then from ONNX convert to Openvino IR. So I converted the model from torch to ONNX

 

# Export the model to ONNX model
batch_size = 1
x = torch.randn(1,3,1080,1080)
model.eval()
torch_out = model(x)
torch.onnx.export(
    model,
    x,
    "cocoa_diseasis_model.onnx",
    export_params=True,
    opset_version=11,
    verbose=False,
    do_constant_folding=True,
    input_names = ['input'],
    output_names = ['output'],
    dynamic_axes={'input' : {0 : 'batch_size'}, 'output' : {0 : 'batch_size'}}
)

 

but trying to get the IR of that model generates the following error.

 

py ./mo_onnx.py --input_model input_model\model.onnx --transformations_config ./extensions/front/onnx/mask_rcnn.json
[ ERROR ] -------------------------------------------------
[ ERROR ] ----------------- INTERNAL ERROR ----------------
[ ERROR ] Unexpected exception happened.
[ ERROR ] Please contact Model Optimizer developers and forward the following information:
[ ERROR ] Exception occurred during running replacer "ONNXMaskRCNNReplacement (<class 'extensions.front.onnx.mask_rcnn_conversion.ONNXMaskRCNNTransformation'>)": Attempt to access node 2751 that not in graph
[ ERROR ] Traceback (most recent call last):
File "C:\Program Files (x86)\Intel\openvino_2021\deployment_tools\model_optimizer\mo\utils\class_registration.py", line 278, in apply_transform
for_graph_and_each_sub_graph_recursively(graph, replacer.find_and_replace_pattern)
File "C:\Program Files (x86)\Intel\openvino_2021\deployment_tools\model_optimizer\mo\middle\pattern_match.py", line 46, in for_graph_and_each_sub_graph_recursively
func(graph)
File "C:\Program Files (x86)\Intel\openvino_2021\deployment_tools\model_optimizer\mo\front\tf\replacement.py", line 36, in find_and_replace_pattern
self.transform_graph(graph, desc._replacement_desc['custom_attributes'])
File "C:\Program Files (x86)\Intel\openvino_2021\deployment_tools\model_optimizer\extensions\front\onnx\mask_rcnn_conversion.py", line 38, in transform_graph
insert_ExperimentalDetectronROIFeatureExtractor2(graph, replacement_descriptions)
File "C:\Program Files (x86)\Intel\openvino_2021\deployment_tools\model_optimizer\extensions\front\onnx\mask_rcnn_conversion.py", line 103, in insert_ExperimentalDetectronROIFeatureExtractor2
old_output_node = Node(graph, replacement_descriptions['ROIFeatureExtractor2_output'])
File "C:\Program Files (x86)\Intel\openvino_2021\deployment_tools\model_optimizer\mo\graph\graph.py", line 25, in __init__
assert node in graph, "Attempt to access node {} that not in graph".format(node)
AssertionError: Attempt to access node 2751 that not in graph

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "C:\Program Files (x86)\Intel\openvino_2021\deployment_tools\model_optimizer\mo\main.py", line 394, in main
ret_code = driver(argv)
File "C:\Program Files (x86)\Intel\openvino_2021\deployment_tools\model_optimizer\mo\main.py", line 356, in driver
ret_res = emit_ir(prepare_ir(argv), argv)
File "C:\Program Files (x86)\Intel\openvino_2021\deployment_tools\model_optimizer\mo\main.py", line 252, in prepare_ir
graph = unified_pipeline(argv)
File "C:\Program Files (x86)\Intel\openvino_2021\deployment_tools\model_optimizer\mo\pipeline\unified.py", line 13, in unified_pipeline
class_registration.apply_replacements(graph, [
File "C:\Program Files (x86)\Intel\openvino_2021\deployment_tools\model_optimizer\mo\utils\class_registration.py", line 328, in apply_replacements
apply_replacements_list(graph, replacers_order)
File "C:\Program Files (x86)\Intel\openvino_2021\deployment_tools\model_optimizer\mo\utils\class_registration.py", line 314, in apply_replacements_list
apply_transform(
File "C:\Program Files (x86)\Intel\openvino_2021\deployment_tools\model_optimizer\mo\utils\logger.py", line 111, in wrapper
function(*args, **kwargs)
File "C:\Program Files (x86)\Intel\openvino_2021\deployment_tools\model_optimizer\mo\utils\class_registration.py", line 302, in apply_transform
raise Exception('Exception occurred during running replacer "{} ({})": {}'.format(
Exception: Exception occurred during running replacer "ONNXMaskRCNNReplacement (<class 'extensions.front.onnx.mask_rcnn_conversion.ONNXMaskRCNNTransformation'>)": Attempt to access node 2751 that not in graph

[ ERROR ] ---------------- END OF BUG REPORT --------------
[ ERROR ] -------------------------------------------------

 

can someone help me to do that?

0 Kudos
8 Replies
IntelSupport
Community Manager
950 Views

Hi Kenfack,

Thanks for reaching out.

Which specific OpenVINO version you are using? Can you share your weight file or model for us to test at our end?

Meanwhile, you can try to use the Model Optimizer parameter from the documentation below to convert your model into Intermediate Representation (IR).

https://docs.openvinotoolkit.org/2021.4/openvino_docs_MO_DG_prepare_model_convert_model_onnx_specific_Convert_Mask_RCNN.html

 

Regards,

Aznie


0 Kudos
kenfack
Beginner
941 Views

Hi @Aznie_Intel , 

Thank you for your reply. I'm using the last version of the Openvino toolkit (v 2021.4.582). 

The Pytorch fine-tuned model

The Converted model to ONNX using opset 11 

The Converted model to ONNX using opset 13

 

Regards,

Bruno

0 Kudos
IntelSupport
Community Manager
916 Views

Hi Bruno,

Can you share the source where you got the model?


Regards,

Aznie


0 Kudos
kenfack
Beginner
911 Views

Hi @IntelSupport ,

This is how I loaded the model, I'm following this tutorial 

num_classes = 4
model = models.detection.maskrcnn_resnet50_fpn(pretrained=True)
in_features = model.roi_heads.box_predictor.cls_score.in_features
model.roi_heads.box_predictor = FastRCNNPredictor(in_features, num_classes)
in_features_mask = model.roi_heads.mask_predictor.conv5_mask.in_channels
hidden_layer = 256
model.roi_heads.mask_predictor = MaskRCNNPredictor(in_features_mask,hidden_layer,num_classes)

Find the notebook here

Regards,

0 Kudos
IntelSupport
Community Manager
872 Views

Hi Bruno,

Thank you for your patience.

After a detailed checking on your model, we found that the issue is from the IF operation that is not yet supported. Meanwhile, for the latest OpenVINO, you can directly use .onnx to run OpenVINO inference, which means you do not require to convert to IR. You can directly use the benchmark app to check your model that already converted from Pytorch to ONNX by using the following command:

 

Python benchmark_app.py - cocoa_diseases_model.onnx

 

I have tested your model with the benchmark app and the error messages clearly state that the IF Operation is not supported. Hence, I would advise you either removed the IF operation or change the IF operation to another operation that is supported based on ONNX* Supported Operators.

 

Regard

Aznie


0 Kudos
kenfack
Beginner
862 Views

Hi @Aznie_Intel , 

happy to hear from you. Thank you for your response. 

  1. How to change or remove the IF operation? Please for a tutorial.
  2. I still need to convert my model into IR because I have a final conversion to do. From IR to Blob. So that the final model can be used in the DeepthAI device (OAK-D)

 

Regards,

Bruno

0 Kudos
IntelSupport
Community Manager
801 Views

Hi Bruno,

Unfortunately, we don’t have the official tutorial from the OpenVINO perspective. I would advise you to go through or create a pull request on how to edit the ONNX model on the official Github ONNXruntime.

 

Regards,

Aznie


0 Kudos
IntelSupport
Community Manager
787 Views

Hi Bruno,

This thread will no longer be monitored since we have provided a solution. If you need any additional information from Intel, please submit a new question.


Regards,

Aznie


0 Kudos
Reply