Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6392 Discussions

TOO many reorders in a binary network IR trained from NNCF

Raza__Ghulam_Jilani
1,228 Views
System information (version)
  • OpenVINO = 2021.1.110
  • Operating System / Platform => ubuntu 18.04
  • Problem classification: Model Conversion
  • Framework: NNCF (Pytorch 1.8.1)
  • Model name: Squeezenet (Classification)
  • CPU: Intel core i7 9700k
Detailed description

I trained a squeezenet classification model using given training pipeline script in nncf examples with XNOR binarization as compression method, keeping first conv layer as fp32 and rest of the net in binary, converted to onnx and then to IR via model optimizer and ran benchmarks with the OV's benchmark tool and found that in addition to fake quantize, there are reorders happening before every binary conv layer execution which are costly and essentially taking away the speed up that could be achieved through binarization as a result, the FPS are even lower that a full FP32 model (benchmark report attached).

As you can see below before every binary conv, a reorder is happening from nchw8c to nhwc, although as per docs of binary conv here, the binary conv layer can accept NCHW format.

 

binary_net_benchmark.png

 

Concern is, why the model optimizer/Inferece Engine is adding reorders before every bin conv and is there a way to avoid these?
Detailed benchmark counters for full FP32 model and binary model are attached for reference.

 

 

I've also created an issue for this on the github repo, posting here as well for better reach/response

 

Regards,

Raza

Labels (1)
0 Kudos
4 Replies
Zulkifli_Intel
Moderator
1,201 Views

Hello Reza Ghulam Jilani,


Greetings to you.


We are investigating this issue. We notice that you open another ticket on Github #5817, we will treat these cases as one.


Regards,

Zulkifli 


0 Kudos
Raza__Ghulam_Jilani
1,191 Views

Please find below the link to relevent onnx model files for your referece:

https://drive.google.com/drive/folders/1EzkohZOjGg-Cul0IoLok9QJu1NO0GMQ2?usp=sharing

0 Kudos
Zulkifli_Intel
Moderator
1,150 Views

Hello Ghulam Jilani Raza,


Thank you for your patience. The dev team is investigating the issue and will provide the finding in this GitHub thread #5817.


Regards,

Zulkifli


0 Kudos
Zulkifli_Intel
Moderator
1,096 Views

Hello Ghulam Jilani Raza,

 

We will continue the investigation on GitHub. Therefore, we will close this thread and focus on GitHub:

https://github.com/openvinotoolkit/openvino/issues/5817

 

Hence, this thread will no longer be monitored by Intel. If you need any additional information from Intel, please submit a new question.

 

Sincerely,

Zulkifli 


0 Kudos
Reply