Qunatization of Pretrained model

[Shravanthi]

Hi,

I am trying to quantize a pretrained model using NNCF (Neural Network Compression Framework) 8-bit quantization to optimize a PyTorch/ONNX/Tensorflow models for the high-speed inference. I have used nncf.quantize for the quantization and have prepared calibration data using  torch.utils.data.DataLoader(...), I am facing issues in quantizing a model, irrespective of model used I get same TypeError always. Below is the sample code and error screenshot. Could you please help to resolve this issue.

Regards,

Shravanthi J

[Aznie_Intel]

Hi Shravanthi J,

Thanks for reaching out.

From your snippet code, I can't see any Ellipsis usage. In Python, Ellipsis is a reserved keyword. You can't name your variables, classes, functions, or methods with the same exact name. Ellipsis is a built-in singleton in Python just like other keywords like pass, True, False, etc. In function bodies, Ellipses work just like pass. You may refer to Ellipsis in Python (*This is an external link and is not maintained by Intel).

I would advise you to follow the Quantizing Models guide to apply model quantization in OpenVINO.Else, you may try command-line interface of POT which is designed to quantize models from OpenVINO Model Zoo. 

Hope this helps.

Regards,

Aznie

[Shravanthi]

Hi,

I tried to quantize model using CLI POT but we need yaml configurations so could not proceed further as I am trying to quantize pretrained models from mlperf or other resources. Howevere, I quantized model using nncf and calibration dataset with one image, I was able to compress weights, Now I want to check accuracy of the pretrained quantized model. Are there any ways to check accuracy without training the model from scratch ?  

Regards,

Shravanthi J

[Aznie_Intel]

Hi Shravanthi J,

You may use Accuracy Checker in OpenVINO. Refer to the Deep Learning accuracy validation framework to install and configure the Accuracy Checker.

Regards,

Aznie

[Aznie_Intel]

Hi Shravanthi J,

This thread will no longer be monitored since we have provided a solution. If you need any additional information from Intel, please submit a new question. 

Regards,

Aznie