Research On Channel Pruning Methods For Deep Convolutional Neural Networks

Deep convolutional neural networks have achieved excellent performance beyond traditional methods in areas such as computer vision and natural language processing.However,with the increase in network size and the emergence of various complex structures,it is more difficult to apply and deploy convolutional networks in scenarios with limited memory and computing power such as autonomous driving,wearable devices,and intelligent robots,which limit the applications and development of convolutional networks and related tasks.Therefore,it is necessary and urgent to conduct theoretical analysis on the compression of deep convolutional neural networks and build efficient compression models without suffer from accuracy loss.At present,some studies have revealed the obvious parameter redundancy in deep convolutional networks,which builds a theoretical basis for network compression.Moreover,there is still much room for improvement in the existing methods in terms of parameter importance evaluation criteria,compression efficiency,and accuracy loss.Based on the analysis and summary of domestic and foreign network compressing methods,this dissertation focuses on the channel pruning technique of convolutional neural networks for in-depth research.The main work accomplished is summarized as follows.1)A channel pruning method based on the magnitude of intermediate feature attention is proposed.The intermediate activation of the feature attention module is used to evaluate the importance of the channels,and then a threshold is set to remove the unimportant feature maps and their corresponding filters in the network.The problem that the existing magnitude-based pruning tend to ignore the role of parameters in the whole network is solved.In addition,a new pruning strategy is proposed for the basic residual modules to keep the number of input and output channels of residual modules consistent.by uniformly pruning the residual modules contained in the same stage.2)A channel clustering pruning method based on the particle swarm optimization is proposed.Firstly,the channels are clustered and pruned based on the cosine similarity between feature maps,then the pruned sub-networks are expanded into candidate populations.Finally,the particle swarm optimization algorithm is used to search for the channel number configuration with better performance within a certain compressing rate.In this way,the problem that existing network pruning methods rely more on human intervention is solved,and automatic pruning is realized.3)A global balanced iterative pruning method combined knowledge transfer and adversarial game is proposed.By analyzing the magnitude distribution of the output feature maps in different layers,it reveals that the existing pruning methods have the disadvantages of unbalanced pruning rate of the number of parameters and floating-point operations.In this scenario,this dissertation proposes a new global balanced iterative pruning strategy based on maximum normalization of feature maps.In addition,we further design a performance recovery strategy during pruning intervals to achieve accurate pruning by pruning-optimization iterations to ensure that the network has good accuracy in each pruning process.In this way,the incidence of mis-pruning caused by single-stage pruning is reduced.