Intel® Distribution of OpenVINO™ Toolkit
Community support and discussions about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all things computer vision-related on Intel® platforms.

ONNX Model With Custom Layer

SalimNamvar
Novice
1,277 Views

Hi,

 

I need to convert my Pytorch ONNX model to OpenVino optimized model. The ONNX model has a custom layer of DCNv2 (Deformable Convolution). There is not any tutorial for converting ONNX models. Can you please help me with a tutorial?

 

Thanks

0 Kudos
1 Solution
Sahira_Intel
Moderator
1,107 Views

Hi Salim,

Here is the article for how to add support for an unsupported layer. In the example, they are using the ONNX Framework and adding support for the ReduceL2 Layer. This should give you a good idea of the process, but if you do have any questions please let me know.

Thanks so much for your patience and I hope this is helpful.

Best Regards,

Sahira 

View solution in original post

13 Replies
Iffa_Intel
Moderator
1,249 Views

Greetings,


You may refer to this official documentation regarding the ONNX conversion: https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_convert_model_Convert_Mode...


Please help to ensure that your topology/model is supported based from the documentation listings.


Sincerely,

Iffa



SalimNamvar
Novice
1,244 Views

This page does not show how to register a custom layer in ONNX models. 

Iffa_Intel
Moderator
1,235 Views

Please note that we only support things that are related with intel OpenVINO implementation only.

If you are asking about OpenVINO pre-trained model or IR convertion from custom model to be used with OpenVINO, optimization, or anything else related with OpenVINO, then we can help.

If you are creating your own custom model and attempting to train them according to what you desire, then it is not related with OpenVINO.


However, you may try to study this:https://github.com/Russzheng/ONNX_custom_layer


Plus, this is how to convert a complete ONNX model to IR and use it with OpenVINO: https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_convert_model_Convert_Mode...


Basically, from ONNX file you will have 2 files (.bin and .xml )after the conversion. These are known as the IR files.



Sincerely,

Iffa


SalimNamvar
Novice
1,231 Views

Thanks,

I don't need to train my model. I have my own model that is trained in Pytorch. I converted my model to ONNX to be loaded in OpenVino. My model has custom layers of DCNv2(Deformable Convolutional Networks). Now I just need to load my ONNX model in OpenVino to be converted to IR files and then use it in inference time. 

I need to know how to register a new custom layer in ONNX models for OpenVino. At the link of https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_customize_model_optimizer_... you have some instructions for custom layers, but they are for Tensorflow, Caffe, and MXNet. There is not any solution for registering a new custom layer.

When I use your instruction for loading ONNX models, I get this error: [so, I must register my custom layer]

[ ERROR ] Cannot infer shapes or values for node "DCNv2_183".
[ ERROR ] There is no registered "infer" function for node "DCNv2_183" with op = "DCNv2". Please implement this function in the extensions.
For more information please refer to Model Optimizer FAQ, question #37. (https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html?q...)
[ ERROR ]
[ ERROR ] It can happen due to bug in custom shape infer function <UNKNOWN>.
[ ERROR ] Or because the node inputs have incorrect values/shapes.
[ ERROR ] Or because input shapes are incorrect (embedded to the model or passed via --input_shape).
[ ERROR ] Run Model Optimizer with --log_level=DEBUG for more information.
[ ERROR ] Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.PartialInfer.PartialInfer'>): Stopped shape/value propagation at "DCNv2_183" node.
For more information please refer to Model Optimizer FAQ, question #38. (https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html?q...)

 

With kind regards,

Salim

Iffa_Intel
Moderator
1,194 Views

Greetings,


Currently the DCNV2 layer is unsupported in ONNX.

You will needs to implement it as a custom layer

We are working on publishing an article to show how to do this in ONNX. Will update shortly.


Sincerely,

Iffa


SalimNamvar
Novice
1,190 Views

Dear Iffa,

Thank you so much for the update,

 

Yes, the DCNV2 layer is a custom layer and I need to implement it as a custom layer.

Could you please share the article here when it is completed?

 

Thank you so much,

With kind regards,

Salim

Sahira_Intel
Moderator
1,133 Views

Hi Salim,

I am still in the process of publishing the documentation for implementing a custom layer in ONNX. I will update you as soon as it is published, I apologize for the delay.

 

Best Regards,

Sahira 

Sahira_Intel
Moderator
1,108 Views

Hi Salim,

Here is the article for how to add support for an unsupported layer. In the example, they are using the ONNX Framework and adding support for the ReduceL2 Layer. This should give you a good idea of the process, but if you do have any questions please let me know.

Thanks so much for your patience and I hope this is helpful.

Best Regards,

Sahira 

View solution in original post

SalimNamvar
Novice
1,090 Views

Hi Sahira,

Thank you so much for your consideration. 

Yes, it is so helpful for me. I'll take a look at it and let you if there is any question.

Kind regards,

Salim

lenbus
Beginner
1,073 Views

Hey Salim,

have the exact same problem here. However I did not manage to get the DCNv2-layer to be transformed properly. Would you please let me know, how you have registered that one?

 

Best regards from Germany

lenbus

SalimNamvar
Novice
1,066 Views

Dear Lenbus,

 

I've not tried the new documentation yet. If I find how to register the DCNv2 layer, I'll let you know about that. If you register it properly, please let me know too.

Thanks, with kind regards,

Salim

Iffa_Intel
Moderator
989 Views

Greetings,


Intel will no longer monitor this thread since this issue has been resolved. If you need any additional information from Intel, please submit a new question


Sincerely,

Iffa



lenbus
Beginner
913 Views

Oh really? Can you tell me, where to find that one? Currently I can't see DCNv2 anywhere in the supported ONNX- layers. I would be really thankful if this finally has support.

 

Regards from Germany

Lenbus

Reply