Research On Structured Pruning Algorithm Of Convolution Neural Networks

With the rapid development of deep learning,the depth and width of deep neural network are increasing.The actual application equipment is limited by the calculation and storage of neural network,which makes the trained network model difficult to use in mobile devices and embedded devices.Pruning is a simple and efficient compression method.By pruning a neuron or filter in the deep neural network,the computational complexity of the network is reduced.The existing pruning methods can be divided into structured pruning and unstructured pruning.However,because unstructured pruning needs special hardware and deep learning library support,it is difficult to accelerate the existing hardware devices.Therefore,this paper focuses more on the research of structural pruning,reducing the size of the network model without destroying the network structure.In order to select the redundant filter in the network more accurately,the paper proposes two different pruning methods:1)A dynamic network pruning method via filter attention mechanism and feature scaling coefficientThe existing pruning method considers more the influence of convolution layer on the filter,but ignores the influence of batchnormalization layer on the selection of filter,which can not guarantee the accuracy of the selection of redundant filter.In order to solve this problem,using the dynamic network pruning method of double-layer information of convolution layer and batchnormalization layer,the influence of convolution layer and batchnormalization layer on the selection of redundant filter is considered respectively.The redundant filter is selected more accurately,and the redundant filter is pruned to improve the prediction accuracy of the network.2)A structural pruning method of dynamic network via layer fusion characteristic coefficientOn the basis of the above methods,from considering the influence of convolution layer and batchnormalization layer on the selection of redundant filters separately to considering the influence of convolution layer and batchnormalization layer on the selection of redundant filters comprehensively,the expressions of convolution layer and batchnormalization layer are formalized into full connection form.Multi dynamic parameter variables are introduced to dynamically determine the redundant filters of different layers,which effectively reduces the size of the model and the amount of computation of the network.Experiments show that the compressed network has high precision,no matter in residual network or lightweight network.Even in a certain range of compression rate,the compressed network exceeds the original network precision.