Output mismatch with original mxnet models

[juierror]

I try to convert mxnet model to  OpenVINO IR, but some operation didn't support (_contrib_div_sqrt_dim, _contrib_arange_like, batch_dot, LayerNorm), so I try https://docs.openvinotoolkit.org/latest/openvino_docs_HOWTO_Custom_Layers_Guide.html this link and get the model can convert to IR and run now. The problem is the output is mismatch with the original mxnet models. 

At first I think it was my implementation on those 4 layers is not correct, but I already separate check output from each layer with original mxnet model. The result is the same. Thus, I debug found that in original model, it have 12 LayerNorm, but when run with OpenVINO, it run only 10 LayerNorm and make output mismatch. (The weird thing is I check OpenVINO xml file and see 12 LAyerNorm). It seem like some part of the models didn't run.

OpenVINO didn't run LayerNorm in this part, so the work around method I use is get the Const value from mxnet and force Const as OpenVINO inputs. With this method, I get same result from OpenVINO and original mxnet models. However, it seem like this work around may hard to use on production.

I will attach the xml file which get the same result and didn't get the same result and some code which I add for support LayerNorm. (I cannot provide weight file).

P.S. I use openvino 2021.4 version

[IntelSupport]

Hi Juierror,

Thanks for reaching out.

As you are already aware, your custom MXNET model has an unsupported framework layer that cannot be executed by OpenVINO.

These operation symbols that you listed (_contrib_div_sqrt_dim, _contrib_arange_like, batch_dot, LayerNorm) are indeed not in the list of supported MXNET frameworks layers.

Great to know that you are successfully enabled in adding custom operation in MO guides to these unsupported operations and perform a workaround on the mismatched results between the original & IR models.

We are totally on the same page with you that these workarounds might not be well-suited for production mode.

We always welcomed customers' inputs on improving our solution and this can be treated as a feature request that we can help you to channel them to our development team for potential enablements.

Appreciate if you could provide your confirmation that you would like us to bring this forward.

Regards,

Aznie

[juierror]

Thank you for your kind reply, it would be great if someone can help on this topic.

[IntelSupport]

Hi Juierror,

I will escalate this feature request to our engineering team for further action. However, it is not feasible for an immediate fix since we will need a period of time for validation before being released.

Regards,

Aznie

[juierror]

Thank you very much. I really appreciate it.

[IntelSupport]

Hi Juierror,

Thank you for your patience. We are working on this feature request and we need your trained model to further investigate this. Could you please share your trained model for us to further understand the request?

Regards,

Aznie

[juierror]

Thank you for your reply. I cannot provide the real trained model, but I can provide the random weight of model. I will try to prepare random weight and the step to reproduce the problem, then send it to you in this week.

Thank you again.

[juierror]

Hi Aznie,

This attach file contains the random weight models and README how to reproduce output mismatch (also how to make output match with the mxnet model). Feel free to tell me if you need anything more. Thank you very much.

[IntelSupport · Accepted solution]

Hi Juierror,

For your information, we had received an update from our developers that the changes to add these MXNet operations are targeted to be released on OpenVINO 2022.1.

Regards,

Aznie

[juierror]

That would be nice. Thank you very much.

[IntelSupport]

Hi Juierror,

This thread will no longer be monitored since this issue has been resolved. If you need any additional information from Intel, please submit a new question.

Regards,

Aznie