- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The following is sequence of quantization
# Create a quantized model from a pre-trained FP32 model and configuration object.
compress_ctrl, compress_model = create_compressed_model(
model, nncf_config
)
#warnings.filterwarnings("ignore", category=TracerWarning) # Ignore export warnings
#warnings.filterwarnings("ignore", category=UserWarning)
compress_ctrl.export_model(str(outdir)+"model_int8.onnx")
When I quantiza a model, I'd like to use option: operator_export_type=torch.onnx.OperatorExportTypes.ONNX_ATEN_FALLBACK or opset_version becuase I'd like to avoid error: RuntimeError: Unsupported: ONNX export of operator adaptive_avg_pool2d, since output size ...
When I used the option above when converting FP32 model to onnx, the option allowed me to convert w/o error. I'd like to use the same option when converting INT8 model.
Is this acceptable? I cheked a function compress_ctrl.export_model, but I could not find a proper argument....
best regards,
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi timosy,
Yes, you can do so as well. The current positioning of NNCF is such that we target OpenVINO™ as the inference framework for the NNCF-created models and do not guarantee functionality for the other inferencing frameworks.
Regards,
Peh
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi timosy,
Thanks for reaching out to us.
Unfortunately, I also not aware whether it is possible to have the operator_export_type option to be added to the compress_ctrl.export_model as it only takes a single parameter specifying the path to the output ONNX model. Anyway, you can post NNCF related questions on NNCF GitHub.
Sincerely,
Peh
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Dear Peh_Intel
Thanks for your comment.
I'm also testing an alternate procedure, it means quantizing a model under onnx framework, then convert it to IR model, where NNCF is not necessary...
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi timosy,
Yes, you can do so as well. The current positioning of NNCF is such that we target OpenVINO™ as the inference framework for the NNCF-created models and do not guarantee functionality for the other inferencing frameworks.
Regards,
Peh
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi timosy,
This thread will no longer be monitored since we have provided answers and suggestions. If you need any additional information from Intel, please submit a new question.
Regards,
Peh
![](/skins/images/B7BA37F9CE4B52FF7CAE1FB0388839C8/responsive_peak/images/icon_anonymous_message.png)
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page