Ridurre - Filter Pruning in Deep Convolutional Networks

Pruning is the process when we try to shrink a network by removing the not so significant/redundant filters.

This package is a mini-framework which you can easily use on your existing models and also you can define your own pruning methods without any struggle.

pruning framework diagram

Install

pip install ridurre

Example Results

These results were achieved with the example provided:

training with pruning pruning

Usage

Define you own pruning method

You can make your own pruning method by creating a new class which has the parent BasePruning. There is only 1 thing you should take care and that the implementation of the run_pruning_for_conv2d_layer function.

For an example just take a look at the RandomFilterPruning code.

Use an already existing method

Check out the example/model_pruning_example.py for a simple but extensive tutorial

Callbacks

You will need to define 2 callbacks for the pruning:

Pruning

You will need to select which pruning method you would like to use. In this example I will use the KMeans pruning

import ridurre

# Create the model
model = build_model(...)

# Define compile callback
def compile_my_model(model):
    model.compile(...)

# Compile with your callback (of course you can use different compilation for this train and the pruning)
compile_my_model(model)

# Train if you would like to start from a better position
model.fit(...)

# Define finetuning callback
def finetune_my_model(model, initial_epoch, finetune_epochs):
    model.fit(..., epochs=finetune_epochs, initial_epoch=initial_epoch)

# We can start pruning
pruning = ridurre.KMeansFilterPruning(0.9,
                                             compile_my_model,
                                             finetune_my_model,
                                             6,
                                             maximum_pruning_percent=0.4,
                                             maximum_prune_iterations=12)
model, _ = pruning.run_pruning(model)

At the end of the pruning step, you will have a trained and pruned model which you can use. I can recommend to train your model after the pruning for just a little longer as an extra step towards accuracy.

Future work

Papers

[1] Filter Level Pruning Based on Similar Feature Extraction for Convolutional Neural Networks

[2] Demystifying Neural Network Filter Pruning

About

Gábor Vecsei