ONNX Model With Custom Layer

[SalimNamvar]

Hi,

I need to convert my Pytorch ONNX model to OpenVino optimized model. The ONNX model has a custom layer of DCNv2 (Deformable Convolution). There is not any tutorial for converting ONNX models. Can you please help me with a tutorial?

Thanks

[Iffa_Intel]

Greetings,

You may refer to this official documentation regarding the ONNX conversion: https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_ONNX.html

Please help to ensure that your topology/model is supported based from the documentation listings.

Sincerely,

Iffa

[SalimNamvar]

This page does not show how to register a custom layer in ONNX models. 

[Iffa_Intel]

Please note that we only support things that are related with intel OpenVINO implementation only.

If you are asking about OpenVINO pre-trained model or IR convertion from custom model to be used with OpenVINO, optimization, or anything else related with OpenVINO, then we can help.

If you are creating your own custom model and attempting to train them according to what you desire, then it is not related with OpenVINO.

However, you may try to study this:https://github.com/Russzheng/ONNX_custom_layer

Plus, this is how to convert a complete ONNX model to IR and use it with OpenVINO: https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_ONNX.html

Basically, from ONNX file you will have 2 files (.bin and .xml )after the conversion. These are known as the IR files.

Sincerely,

Iffa

[SalimNamvar]

Thanks,

I don't need to train my model. I have my own model that is trained in Pytorch. I converted my model to ONNX to be loaded in OpenVino. My model has custom layers of DCNv2(Deformable Convolutional Networks). Now I just need to load my ONNX model in OpenVino to be converted to IR files and then use it in inference time. 

I need to know how to register a new custom layer in ONNX models for OpenVino. At the link of https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_customize_model_optimizer_Customize_Model_Optimizer.html you have some instructions for custom layers, but they are for Tensorflow, Caffe, and MXNet. There is not any solution for registering a new custom layer.

When I use your instruction for loading ONNX models, I get this error: [so, I must register my custom layer]

[ ERROR ] Cannot infer shapes or values for node "DCNv2_183".
[ ERROR ] There is no registered "infer" function for node "DCNv2_183" with op = "DCNv2". Please implement this function in the extensions.
For more information please refer to Model Optimizer FAQ, question #37. (https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html?question=37#question-37)
[ ERROR ]
[ ERROR ] It can happen due to bug in custom shape infer function <UNKNOWN>.
[ ERROR ] Or because the node inputs have incorrect values/shapes.
[ ERROR ] Or because input shapes are incorrect (embedded to the model or passed via --input_shape).
[ ERROR ] Run Model Optimizer with --log_level=DEBUG for more information.
[ ERROR ] Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.PartialInfer.PartialInfer'>): Stopped shape/value propagation at "DCNv2_183" node.
For more information please refer to Model Optimizer FAQ, question #38. (https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html?question=38#question-38)

With kind regards,

Salim

[Iffa_Intel]

Greetings,

Currently the DCNV2 layer is unsupported in ONNX.

You will needs to implement it as a custom layer

We are working on publishing an article to show how to do this in ONNX. Will update shortly.

Sincerely,

Iffa

[SalimNamvar]

Dear Iffa,

Thank you so much for the update,

Yes, the DCNV2 layer is a custom layer and I need to implement it as a custom layer.

Could you please share the article here when it is completed?

Thank you so much,

With kind regards,

Salim

[Sahira_Intel]

Hi Salim,

I am still in the process of publishing the documentation for implementing a custom layer in ONNX. I will update you as soon as it is published, I apologize for the delay.

Best Regards,

Sahira 

[Sahira_Intel · Accepted solution]

Hi Salim,

Here is the article for how to add support for an unsupported layer. In the example, they are using the ONNX Framework and adding support for the ReduceL2 Layer. This should give you a good idea of the process, but if you do have any questions please let me know.

Thanks so much for your patience and I hope this is helpful.

Best Regards,

Sahira 

[SalimNamvar]

Hi Sahira,

Thank you so much for your consideration. 

Yes, it is so helpful for me. I'll take a look at it and let you if there is any question.

Kind regards,

Salim

[lenbus]

Hey Salim,

have the exact same problem here. However I did not manage to get the DCNv2-layer to be transformed properly. Would you please let me know, how you have registered that one?

Best regards from Germany

lenbus

[SalimNamvar]

Dear Lenbus,

I've not tried the new documentation yet. If I find how to register the DCNv2 layer, I'll let you know about that. If you register it properly, please let me know too.

Thanks, with kind regards,

Salim

[Iffa_Intel]

Greetings,

Intel will no longer monitor this thread since this issue has been resolved. If you need any additional information from Intel, please submit a new question

Sincerely,

Iffa

[lenbus]

Oh really? Can you tell me, where to find that one? Currently I can't see DCNv2 anywhere in the supported ONNX- layers. I would be really thankful if this finally has support.

Regards from Germany

Lenbus