Study Of Accelerating Convolutional Neural Networks Based On Channel Pruning

Convolution neural networks(CNNs)have achieved significant success in computer vision and speech recognition,meanwhile the depth and width of CNNs are increasing further,which lead to a sharp increase in computational complexity of networks.At present,CNNs have a huge number of parameters,the huge memory and high computational costs required by these networks restrict application of deep learning on mobile devices with limited resources.This paper introduces a new channel pruning method to reduce the floating-point operations(FLOPs).By removing the less important channels in convolution kernels,a smaller CNN model is obtained,the effect of accelerating and compressing CNNs can be achieved.In this paper,in order to reduce the FLOPs of CNNs,channel pruning method is studied under the premise that network performance degradation can be neglected.The main contributions are summarized as follows:(1)For two common convolutional neural networks with different structures,i.e.convolutional neural networks without shortcut connections and residual networks with shortcut connections,the process of channel pruning for these two networks is given,and the reduction in network FLOPs by removing channels for convolutional layer is discussed.(2)A new pruning criterion is proposed based on mean gradient for feature maps.The criterion measures the importance of each channel by mean gradient for its single feature map.The smaller mean gradient for a feature map is,the smaller influence on networks performance will be when its corresponding channel is removed,that is,the convolutional layer of the corresponding channel is less sensitive to be pruned.Compared with the existing pruning criteria,the accuracy of the pruned model with mean gradient criterion is relatively stable,and it has less impact on network performance.(3)In view of the problem that pruning on layer-by-layer strategy is extremely time-consuming and global pruning strategy cannot significantly reduce networks FLOPs,hierarchical global pruning strategy is introduced.According to the pruning criterion,global pruning strategy is adopted between convolutional layers with similar sensitivity.Hierarchical global pruning strategy actively learns pruning ratio for each convolutional layer,which avoids serious damage to network performance when the fixed pruning ratio of convolution layer is too high.Meanwhile hierarchical global pruning strategy ensures similar pruning ratios between each convolutional layer and overall network,which achieve a significant reduction in network FLOPs.Experimental results show that VGG-16 network pruned by channel pruning on CIFAR-10 achieves5.64× reduction in FLOPs with less than 1% decrease in accuracy.Meanwhile Res Net-110 network pruned on CIFAR-10 achieves 2.48× reduction in FLOPs and parameters with only 0.08% decrease in accuracy.