Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6418 Discussions

How to convert a pruned Keras model to IR without losing the model sparsity?

publio
New Contributor I
463 Views

Hello! I have a Keras DNN model that I am trying to compare with Google's Coral Edge TPU. What I am doing is after training i am using tensorflow pruning tool to add sparsity my model and compress it and then converting it to OpenVino format and TFLite. 

 

Although, for the Edge TPU case, Tensorflow has a pruning-quantization-aware training method that lets us quantize the model before converting the model to the accelerator required file format. 

 

My model has a test accuracy of 91% before converting to NCS2, after pruning I get 91.33%. However, when reducing the model precision to float16 for the NCS2 device, I get 91%  test accuracy.

Whereas on Edge TPU, I get 93.33% using the PQAT technique.   From what I deduced, is that the OpenVino tool will not keep the sparsity from a pruned tensorflow model when reducing the model precision. I believe my trained model may benefit from pruning + precision reduction in terms of test accuracy. 

 

I'd like to know if there is any way to preserve the model sparsity after I use pruning and convert it to IR format?

0 Kudos
2 Replies
Wan_Intel
Moderator
429 Views

Hi Publio,

Thanks for reaching out to us.

 

Based on your description, OpenVINO™ Model Optimizer does not preserve the model sparsity that was pruned with preserving quantization-aware training (PQAT). We regret to inform you that PQAT is not validated with OpenVINO™.

 

For your information, Neural Network Compression Framework (NNCF) provides a suite of advanced algorithms for neural network optimization in OpenVINO™ with minimal accuracy drop. It supports various compression algorithms that support TensorFlow as shown as follows:

 

Hope it helps.

 

 

Regards,

Wan

 

0 Kudos
Wan_Intel
Moderator
406 Views

Hi Publio,

Thanks for your question.

 

This thread will no longer be monitored since we have provided a suggestion. 

If you need any additional information from Intel, please submit a new question.

 

 

Regards,

Wan


0 Kudos
Reply