topic Re: The quantized INT8 onnx models fails to load with invalid model error in AI Tools from Intel

The quantized INT8 onnx models fails to load with invalid model error

Anand_Viswanath — Thu, 18 Aug 2022 11:52:35 GMT

Hi,

I have quantized my ONNX FP32 model to ONNX INT8 model using Intel's Neural Compressor.

When I try to load the model to run inference, it fails with an error saying Invalid model.

Error:

File "testOnnxModel.py", line 178, in inference
session = ort.InferenceSession(onnx_file, sess_options=sess_options, providers=['CPUExecutionProvider'])
File "/root/anaconda3/envs/versa_benchmark/lib/python3.7/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 335, in __init__
self._create_inference_session(providers, provider_options, disabled_optimizers)
File "/root/anaconda3/envs/versa_benchmark/lib/python3.7/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 370, in _create_inference_session
sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Load model from rappnet2_int8.onnx failed:This is an invalid model. Error: two nodes with same node name (model/rt_block/layer_normalization/moments/SquaredDifference:0_QuantizeLinear).

I am able to quantize and work on the same model's tensorflow version.

Please help on this.

TIA,

Anand Viswanath A

Re:The quantized INT8 onnx models fails to load with invalid model error

DiyaN_Intel — Fri, 19 Aug 2022 12:40:21 GMT

Hi,

Thank you for posting in Intel Communities.

Could you please share the following details?

1. Sample reproducer code

2. exact steps and the commands used

3. OS details.

Regards,

Diya

Re: Re:The quantized INT8 onnx models fails to load with invalid model error

Anand_Viswanath — Mon, 22 Aug 2022 10:00:24 GMT

Hi,

Please find the details below,

Code :

Config :

Steps:

Execute the python script with the quantization code.

The script takes ONNX FP32 model which is converted from TF Frozen FP32 model using TF2onnx.

The output INT8 ONNX model when loaded for inferencing, gives the above error

OS:

Linux sdp 5.4.0-110-generic #124-Ubuntu

Regards,

Anand Viswanath A

Re: The quantized INT8 onnx models fails to load with invalid model error

DiyaN_Intel — Mon, 29 Aug 2022 06:53:49 GMT

Hi ,

As discussed privately please share sample reproducer code which was used for inferencing with exact steps followed .

Regards,

Diya

Re: The quantized INT8 onnx models fails to load with invalid model error

DiyaN_Intel — Tue, 06 Sep 2022 06:05:35 GMT

Hi,

We have not heard back from you.

Could you please share the sample reproducer code which was used for inferencing with exact steps followed so that we can try to fix the issue from our end?

Regards,

Diya

Re:The quantized INT8 onnx models fails to load with invalid model error

DiyaN_Intel — Tue, 13 Sep 2022 07:04:33 GMT

Hi,

We have not heard back from you. This thread will no longer be monitored by Intel. If you need further assistance, please post a new question.

Thanks and Regards,

Diya