Pruning-Based Compression Method For Convolutional Neural Network

As a classical model in deep learning,convolutional neural network(CNN)has achieved a series of research results in various applications such as image classification,object detection and semantic segmentation.However,as the scale and complexity of problem increasing,CNN's parameters and calculation complexity increase exponentially.It demands better training environment and equipment.The promising performance is accompanied by significant computation cost and memory cost.It is difficult to be applied to resource-constrained devices such as mobile phones and embedded devices.Model compression is an effective method to resolve this problem.Therefore,performing research on CNN model compression is conducive to the application of CNN,and it has theoretical and practical significance.Pruning has been widely used to CNN model compression.The main works of this dissertation are as follows.1.A dynamic pruning method based on improved weight pruningThere is huge parameter redundancy in the deep convolutional neural network,and weight pruning is an effective method to reduce the network complexity.Weight pruning is to prune the weight less than a threshold,but this method is irreversible for the mistakenly pruned weights and has an impact on the final network performance.To address this problem,a dynamic pruning method based on weight pruning is proposed.It first prunes the weights below a threshold,then dynamically updates the importance coefficients of the weights,and dynamically restores the mistakenly pruned weights.Compared with the original weight pruning,the experimental results show that the accuracy loss of the proposed dynamic method is reduced by 0.11% and 0.25% on LeNet-5 ? VGG-16 respectively.2.A mixed pruning method combining weight pruning with filter pruningThe compressed convolutional neural network just using weight or filter pruning alone still exists redundant parameters.To address this problem,a mixed pruning method combining weight pruning with filter pruning is proposed.Firstly,filters having little contribution to the overall performance of the convolutional neural network can be pruned.Secondly,the pruned model is further compressed by dynamic pruning to achieve further model compression.Experiments on MNIST and CIFAR-10 datasets demonstrate that the proposed approach is effective and feasible.Compared with weight pruning or filter pruning,the mixed pruning can achieve higher pruning ratio of the model parameters.For LeNet-5,the proposed method can achieve a pruning rate of 12.90×,with just 0.99% drop in accuracy.For VGG-16,it can achieve a compression rate of 19.13×,incurring 1.32% accuracy loss only.