Research On Filter Prunning Method Of Deep Convolution Neural Network

The efficient inference process of deep neural network(DNN)usually relies on high-performance computing devices,which is difficult to be promoted in the resource-constrained mobile terminals or embedded real-time systems.The neural network pruning technology can reduce the complexity of the network by cutting down the number of network connections,and promote the application of neural networks in the resource-limited environments.Currently,in the deep neural network,redundant neuron pruning algorithm has made great progress,but with two key problems:1.The current pruning algorithm lacks theoretical analysis methods that cannot give a reasonable explanation for the preset pruning ratio.2.Weights or neuron pruning strategies based on the global pruning ratio may lead to an excessive pruning of the middle layer of the neural network.Based on the above deficiencies,this paper studied the estimation of the feature channel pruning ratio and the design of pruning algorithm to solve the above two problems respectively.In detail:1.An effective filter number estimation method based on principal component analysis is proposed to explain the setting of pruning ratio and narrow the search range of pruning ratio;2.A new hierarchical pruning algorithm is proposed,which can effectively remove redundant middle layer and neurons without sacrificing network performance.This method is based on the cross entropy between the adjacent two layers to identify and eliminate the redundant layer,which can avoid the problem of excessive pruning in the middle layer.In this paper,several experiments are designed to verify the pruning effect of the proposed method on five kinds of advanced convolutional neural network(CNN)structures.Compared with others,the models used our pruning method can achieve the basically the same results for the classification task at a smaller pruning ratio;and even at a larger pruning ratio,the classification performance of pruned models obtained by the proposed method can still be maintained at a higher level.The experimental results show that the PCA-based effective filter number estimation method proposed in this paper can replace experience and assist existing pruning algorithms to set the filter pruning ratio.