Intel® Distribution of OpenVINO™ Toolkit
Community support and discussions about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all things computer vision-related on Intel® platforms.

Output mismatch with original mxnet models

juierror
Novice
764 Views

I try to convert mxnet model to  OpenVINO IR, but some operation didn't support (_contrib_div_sqrt_dim, _contrib_arange_like, batch_dot, LayerNorm), so I try https://docs.openvinotoolkit.org/latest/openvino_docs_HOWTO_Custom_Layers_Guide.html this link and get the model can convert to IR and run now. The problem is the output is mismatch with the original mxnet models. 

 

At first I think it was my implementation on those 4 layers is not correct, but I already separate check output from each layer with original mxnet model. The result is the same. Thus, I debug found that in original model, it have 12 LayerNorm, but when run with OpenVINO, it run only 10 LayerNorm and make output mismatch. (The weird thing is I check OpenVINO xml file and see 12 LAyerNorm). It seem like some part of the models didn't run.

juierror_0-1627897834995.png

 

OpenVINO didn't run LayerNorm in this part, so the work around method I use is get the Const value from mxnet and force Const as OpenVINO inputs. With this method, I get same result from OpenVINO and original mxnet models. However, it seem like this work around may hard to use on production.

 

I will attach the xml file which get the same result and didn't get the same result and some code which I add for support LayerNorm. (I cannot provide weight file).

 

P.S. I use openvino 2021.4 version

0 Kudos
1 Solution
IntelSupport
Community Manager
542 Views

Hi Juierror,

For your information, we had received an update from our developers that the changes to add these MXNet operations are targeted to be released on OpenVINO 2022.1.


Regards,

Aznie


View solution in original post

10 Replies
IntelSupport
Community Manager
731 Views

Hi Juierror,

Thanks for reaching out.

As you are already aware, your custom MXNET model has an unsupported framework layer that cannot be executed by OpenVINO.

These operation symbols that you listed (_contrib_div_sqrt_dim, _contrib_arange_like, batch_dot, LayerNorm) are indeed not in the list of supported MXNET frameworks layers.

 

Great to know that you are successfully enabled in adding custom operation in MO guides to these unsupported operations and perform a workaround on the mismatched results between the original & IR models.

We are totally on the same page with you that these workarounds might not be well-suited for production mode.

 

We always welcomed customers' inputs on improving our solution and this can be treated as a feature request that we can help you to channel them to our development team for potential enablements.

Appreciate if you could provide your confirmation that you would like us to bring this forward.

 

Regards,

Aznie


juierror
Novice
726 Views

Thank you for your kind reply, it would be great if someone can help on this topic.

IntelSupport
Community Manager
705 Views

 

Hi Juierror,

I will escalate this feature request to our engineering team for further action. However, it is not feasible for an immediate fix since we will need a period of time for validation before being released.

 

Regards,

Aznie


juierror
Novice
699 Views

Thank you very much. I really appreciate it.

IntelSupport
Community Manager
607 Views


Hi Juierror,

Thank you for your patience. We are working on this feature request and we need your trained model to further investigate this. Could you please share your trained model for us to further understand the request?

 

Regards,

Aznie


juierror
Novice
603 Views

Thank you for your reply. I cannot provide the real trained model, but I can provide the random weight of model. I will try to prepare random weight and the step to reproduce the problem, then send it to you in this week.

 

Thank you again.

juierror
Novice
579 Views

Hi Aznie,

This attach file contains the random weight models and README how to reproduce output mismatch (also how to make output match with the mxnet model). Feel free to tell me if you need anything more. Thank you very much.

IntelSupport
Community Manager
543 Views

Hi Juierror,

For your information, we had received an update from our developers that the changes to add these MXNet operations are targeted to be released on OpenVINO 2022.1.


Regards,

Aznie


juierror
Novice
543 Views

That would be nice. Thank you very much.

IntelSupport
Community Manager
532 Views

Hi Juierror,

This thread will no longer be monitored since this issue has been resolved. If you need any additional information from Intel, please submit a new question.


Regards,

Aznie


Reply