Filter pruning statistics, how to monitor

[timosy]

Could anyone teach me how to use "Filter pruning statistics" shown here, https://github.com/openvinotoolkit/nncf/blob/develop/docs/compression_algorithms/Pruning.md#filter-pruning-statistics

Following standard usage of pruning, it does not appear. Are there any paticluar settings to make the stat apper?

I check how the pruning works refering to the information above, because the same pruning happens even if I change "pruning_target". 

Best regards.

[Zulkifli_Intel]

Hello Timosy,

Greetings to you.

You can try to add these parameters to your filter pruning configuration file. You can specify the --log-dir /<path>/ to see the full log file. 

{

   "compression": {

       "algorithm": "filter_pruning",

       "pruning_init": 0.1,

       "params": {

           "schedule": "exponential",

           "pruning_target": 0.4,

           "pruning_steps": 15,

           "filter_importance": "geometric_median",

           "interlayer_ranking_type": "unweighted_ranking",

           "all_weights": true,

           "prune_first_conv": false,

           "prune_downsample_convs": false,

           "prune_batch_norms": true,

           "zero_grad": true

       },

       "ignored_scopes": []

   }

}

Sincerely,

Zulkifli

[timosy]

Thank you for the kind help.

Unfortunately, I can not contoroll the filter pruning.

I changed "pruning_target": 0.75" to 0.50, 0.25. But, I can not see any difference after convrting the model to IR. (The model themselves are well compressed by the pruning. I just could not see the difference dueto the parameter "pruning_target" ) The file sizes (.bin .mapping, .xml) are exsactly same though the  different "pruning_target" was used.

By the way, after converting the model, the model has only a few layers (that's why I set the target layers). The quantization is also included. 

zero_grad is not implemented: Additional properties are not allowed ('zero_grad' was unexpected)

I can also see "log_dir": "./" But, no information on the pruning statistics is logged in the file.

If thre are any suggestions to controlle the filter pruning, I will follow it.

    nncf_config_pruning_quantization_dict = {
    "model": "testnet",
    "num_classes": classes,
    "batch_size": g_batch_size,
    #"log_dir": str(outdir),
    "log_dir": "./",
    "pretrained": True,
    "epochs": 200,
    "input_info": {"sample_size": [1, 3, image_size, image_size] },
    "optimizer": {
        "type": "SGD",
        "base_lr": 0.1,
        "weight_decay": 1e-4,
        "schedule_type": "multistep",
        "steps": [
            20,
            40,
            60,
            80
        ],
        "optimizer_params":
        {
            "momentum": 0.9,
            "nesterov": True
        }
    },
    "compression": [
    {
            "algorithm": "filter_pruning",
            "initializer": {
                "batchnorm_adaptation": {
                    "num_bn_adaptation_samples": 2048,
                    # Number of samples from the training dataset to pass through the model at initialization
                    # in order to update batchnorm statistics of the original model. The actual number of
                    # samples will be a closest multiple of the batch size.
                }
            },
            "pruning_init": 0.1,
            "params": {
                #
                # https://github.com/openvinotoolkit/nncf/blob/develop/docs/compression_algorithms/Pruning.md
                #
                "interlayer_ranking_type": "unweighted_ranking",
                #"interlayer_ranking_type": "learned_ranking",
                # The type of filter ranking across the layers. Can be one of
                # `unweighted_ranking`, `learned_ranking`. `unweighted_ranking` by default.
                "all_weights": True,
                # Whether to prune layers independently (choose filters with the smallest importance
                # in each layer separately) or not. `False` by default.
                "prune_first_conv": True,
                # Whether to prune first Convolutional layers or not. First means that it is a convolutional
                # layer such that there is a path from model input to this layer such that there are no other
                # convolution operations on it. `False` by default
                # (`True` by default in case of 'learned_ranking' interlayer_ranking_type).
                "prune_downsample_convs": True,
                # Whether to prune downsample Convolutional layers (with stride > 1) or not.
                # `False` by default (`True` by default in case of 'learned_ranking' interlayer_ranking_type).
                "prune_batch_norms": True,
                # Whether to nullifies parameters of Batch Norm layer corresponds to zeroed filters of convolution corresponding
                # to this Batch Norm. `True` by default.
                "schedule": "exponential",
                "pruning_target": 0.75,
                "pruning_steps": 10,
                #"filter_importance": "L2",
                #"filter_importance": "L1",
                "filter_importance": "geometric_median"
                # The type of filter importance metric. Can be one of `L1`, `L2`, `geometric_median`. `L2` by default.
            },
            # A list of model control flow graph node scopes to be considered for this operation - functions as a 'allowlist'. Optional.
            # "target_scopes": []
            "target_scopes": [
               # Filter pruning algorithm zeros output filters in Convolutional layers
               "AlexNet/Sequential[features]/NNCFConv2d[0]/conv2d_0",
               "AlexNet/Sequential[features]/NNCFConv2d[3]/conv2d_0",
               "AlexNet/Sequential[features]/NNCFConv2d[6]/conv2d_0",
               #"AlexNet/Sequential[classifier]/NNCFLinear[1]/linear_0",
               #"AlexNet/Sequential[classifier]/NNCFLinear[4]/linear_0",
            ]
       }
       ,{
           "algorithm": "quantization",
           "initializer": {
               # how many samples you use for calibration
               "range": {"num_init_samples": 15000},
               # how many samples you apply for bbatchnorm
               "batchnorm_adaptation": {"num_bn_adaptation_samples": 4000},
           },
       }
    ]
    }

[Zulkifli_Intel]

Hi Timosy,

We are checking this matter with our NNCF team and will revert back to you.

Apart from community, there is an alternative way to reach out to our NNCF team for additional NNCF-specific support in issues - openvino/nncf (GitHub).

Sincerely,

Zulkifli 

[dlyakhov · Accepted solution]

Greetings, @timosy !
Thank you for you interest in NNCF!
First question: what pipeline do you use for model pruning?
If you are using NNCF Pytorch sample, pruning statistics are being printed at the beginning of training  and after each training step . NNCF samples are not dumping statistic to log files, they are printing compression statistics to stdout (your console).
If you are using custom training pipeline, please call statistics method of your compression controller.

Regarding `pruning_target` parameter:
Second question: are you calling `compression_ctrl.step()` and `compression_ctrl.epoch_step()` in you code?
A model is reaching this target according to compression schedule you specified in config.
If you are not calling `compression_ctrl.step()` and `compression_ctrl.epoch_step()` you model will be pruned by `pruning_init` level. Please see training pipeline modification guide for details.

Best regards,
Daniil Lyakhov, NNCF team

[timosy]

Thank you for your comment.

I was originally referring notebook, here https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/112-pytorch-post-training-quantization-nncf where functions such as "epoch_step" is not used. So I missed them.

And sorry, I did not carefully read "Usage.md", I should have read it. Anyway thnak you for pointing out it to me. I will test it.

[Zulkifli_Intel]

Hi Timosy,

Thank you for your question. If you need any additional information from Intel, please submit a new question as this thread is no longer being monitored. 

Sincerely,

Zulkifli