Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.

ONNX Model With Custom Layer

SalimNamvar
Novice
3,641 Views

Hi,

 

I need to convert my Pytorch ONNX model to OpenVino optimized model. The ONNX model has a custom layer of DCNv2 (Deformable Convolution). There is not any tutorial for converting ONNX models. Can you please help me with a tutorial?

 

Thanks

0 Kudos
1 Solution
Sahira_Intel
Moderator
3,471 Views

Hi Salim,

Here is the article for how to add support for an unsupported layer. In the example, they are using the ONNX Framework and adding support for the ReduceL2 Layer. This should give you a good idea of the process, but if you do have any questions please let me know.

Thanks so much for your patience and I hope this is helpful.

Best Regards,

Sahira 

View solution in original post

0 Kudos
13 Replies
Iffa_Intel
Moderator
3,613 Views

Greetings,


You may refer to this official documentation regarding the ONNX conversion: https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_ONNX.html


Please help to ensure that your topology/model is supported based from the documentation listings.


Sincerely,

Iffa



0 Kudos
SalimNamvar
Novice
3,608 Views

This page does not show how to register a custom layer in ONNX models. 

0 Kudos
Iffa_Intel
Moderator
3,599 Views

Please note that we only support things that are related with intel OpenVINO implementation only.

If you are asking about OpenVINO pre-trained model or IR convertion from custom model to be used with OpenVINO, optimization, or anything else related with OpenVINO, then we can help.

If you are creating your own custom model and attempting to train them according to what you desire, then it is not related with OpenVINO.


However, you may try to study this:https://github.com/Russzheng/ONNX_custom_layer


Plus, this is how to convert a complete ONNX model to IR and use it with OpenVINO: https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_ONNX.html


Basically, from ONNX file you will have 2 files (.bin and .xml )after the conversion. These are known as the IR files.



Sincerely,

Iffa


0 Kudos
SalimNamvar
Novice
3,595 Views

Thanks,

I don't need to train my model. I have my own model that is trained in Pytorch. I converted my model to ONNX to be loaded in OpenVino. My model has custom layers of DCNv2(Deformable Convolutional Networks). Now I just need to load my ONNX model in OpenVino to be converted to IR files and then use it in inference time. 

I need to know how to register a new custom layer in ONNX models for OpenVino. At the link of https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_customize_model_optimizer_Customize_Model_Optimizer.html you have some instructions for custom layers, but they are for Tensorflow, Caffe, and MXNet. There is not any solution for registering a new custom layer.

When I use your instruction for loading ONNX models, I get this error: [so, I must register my custom layer]

[ ERROR ] Cannot infer shapes or values for node "DCNv2_183".
[ ERROR ] There is no registered "infer" function for node "DCNv2_183" with op = "DCNv2". Please implement this function in the extensions.
For more information please refer to Model Optimizer FAQ, question #37. (https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html?question=37#question-37)
[ ERROR ]
[ ERROR ] It can happen due to bug in custom shape infer function <UNKNOWN>.
[ ERROR ] Or because the node inputs have incorrect values/shapes.
[ ERROR ] Or because input shapes are incorrect (embedded to the model or passed via --input_shape).
[ ERROR ] Run Model Optimizer with --log_level=DEBUG for more information.
[ ERROR ] Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.PartialInfer.PartialInfer'>): Stopped shape/value propagation at "DCNv2_183" node.
For more information please refer to Model Optimizer FAQ, question #38. (https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html?question=38#question-38)

 

With kind regards,

Salim

0 Kudos
Iffa_Intel
Moderator
3,558 Views

Greetings,


Currently the DCNV2 layer is unsupported in ONNX.

You will needs to implement it as a custom layer

We are working on publishing an article to show how to do this in ONNX. Will update shortly.


Sincerely,

Iffa


0 Kudos
SalimNamvar
Novice
3,554 Views

Dear Iffa,

Thank you so much for the update,

 

Yes, the DCNV2 layer is a custom layer and I need to implement it as a custom layer.

Could you please share the article here when it is completed?

 

Thank you so much,

With kind regards,

Salim

0 Kudos
Sahira_Intel
Moderator
3,497 Views

Hi Salim,

I am still in the process of publishing the documentation for implementing a custom layer in ONNX. I will update you as soon as it is published, I apologize for the delay.

 

Best Regards,

Sahira 

0 Kudos
Sahira_Intel
Moderator
3,472 Views

Hi Salim,

Here is the article for how to add support for an unsupported layer. In the example, they are using the ONNX Framework and adding support for the ReduceL2 Layer. This should give you a good idea of the process, but if you do have any questions please let me know.

Thanks so much for your patience and I hope this is helpful.

Best Regards,

Sahira 

0 Kudos
SalimNamvar
Novice
3,454 Views

Hi Sahira,

Thank you so much for your consideration. 

Yes, it is so helpful for me. I'll take a look at it and let you if there is any question.

Kind regards,

Salim

0 Kudos
lenbus
Beginner
3,437 Views

Hey Salim,

have the exact same problem here. However I did not manage to get the DCNv2-layer to be transformed properly. Would you please let me know, how you have registered that one?

 

Best regards from Germany

lenbus

0 Kudos
SalimNamvar
Novice
3,430 Views

Dear Lenbus,

 

I've not tried the new documentation yet. If I find how to register the DCNv2 layer, I'll let you know about that. If you register it properly, please let me know too.

Thanks, with kind regards,

Salim

Iffa_Intel
Moderator
3,353 Views

Greetings,


Intel will no longer monitor this thread since this issue has been resolved. If you need any additional information from Intel, please submit a new question


Sincerely,

Iffa



0 Kudos
lenbus
Beginner
3,277 Views

Oh really? Can you tell me, where to find that one? Currently I can't see DCNv2 anywhere in the supported ONNX- layers. I would be really thankful if this finally has support.

 

Regards from Germany

Lenbus

0 Kudos
Reply