Unsupported 'Upsample' layer when converting from PyTorch to ONNX to OpenVINO IR Model

[Sharma__Divyanshu]

Hi. I have been trying to convert the RetinaNet model implemented in PyTorch. When converting the model to ONNX, I use OPSET 12, since OPSET 10 and below have different implementations of the 'Resize' operation and it gives very different results from the original implementation.

However, in the list of supported operators by OpenVINO, it only supports the Resize layer for ONNX OPSET 10. What should I do? Is there any way to convert the OPSET 12 version using the OpenVINO model optimizer?

[Iffa_Intel]

Greetings,

First and foremost, before converting you can check which layers are supported by openvino here:

https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Supported_Frameworks_Layers.html

If you take on closer look at the ONNX Supported operator & go to the resize section, you can see the OPSET that are supported.

So far and as you mentioned, only the OPSET version 10 are supported. Hence, it is important to emphasize that OpenVino officially support OPSET10 only.

I highly recommend you to export to OPSET10 if you want to use the OpenVino model optimizer.

Here is some idea on ONNX Opset Version Converter:  https://github.com/onnx/onnx/blob/master/docs/VersionConverter.md

Hope my answer helps!

Sincerely,

Iffa

[Sharma__Divyanshu]

Hi Iffa! Thanks for replying. The issue with using OPSET 10 is that the Resize operation of OPSET 10 does not match PyTorch's implementation of Upsample/Resize. I tried converting the model with OPSET 10 but the results of the converted model are very poor. Could you tell me how I could write a custom implementation? I went through the model optimizer files but could not find the code for Resize. Is it in C++? How do I edit it?

Thanks a lot!

[Iffa_Intel]

Here is some insight for Exporting a Model from PyTorch to ONNX and Running it using ONNX Runtime

https://pytorch.org/tutorials/advanced/super_resolution_with_onnxruntime.html

Form here you can set opset_version=10 (the ONNX version to export the model to)

Then you can proceed to OpenVino ONNX model convertion: https://docs.openvinotoolkit.org/2020.1/_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_ONNX.html

Note that: The Model Optimizer process assumes you have an ONNX model that was directly downloaded from a public repository or converted from any framework that supports exporting to the ONNX format

Sincerely,

Iffa

[Sharma__Divyanshu]

Hi! Thanks for replying

As I said, I can't use opset_version 10 to export the model because the ONNX implementation for the 'Resize' op in opset 10 are very different. I am asking what options I have if I HAVE to use opset 11 and convert this to an OpenVino model.

[Iffa_Intel]

Hi,

Please note that, the current OpenVINO toolkit only supports Resize Opset-10 operation. Our team is aware of such and will consider this for future release.

Thanks & have a great day ahead.

Sincerely,

Iffa

[Sharma__Divyanshu · Accepted solution]

Hi! 

Thanks for your reply. For anyone else who might be having the same issue, I managed to work around it by changing the resize implementation in OpenVino to have a constant scale of 2 (I was lucky because each upscale had a factor of 2).

[Maicus]

Hey

I do have the same problem, that I am not able to use the model optimizer cause of the missing upsampling functionality. In my case I only have 2x upsampling as well. Could you please elaborate how you changed the implementation to get it to work?

Thanks in advance

[Sharma__Divyanshu]

Hey! Sorry for the late reply. I just changed the output shape in the Upsample.py at model-optimizer\extensions\ops to a constant. I've attached the file below.

Cheers!

[Maicus]

Thank you. It works now