Research On Pruning Of Deep Convolutional Neural Network

Deep Convolutional Neural Networks(DCNNs)have achieved remarkable success in image classification,object detection and semantic segmentation.However,this good performance is accompanied by a large amount of storage space and a huge computational expense,because without the support of efficient Graphics Processing Unit(GPU),which brings great difficulties to the deployment of DCNNs on mobile devices with limited computing resources.In order to solve this problem,the use of model pruning to reduce the memory usage and calculation workload of convolutional neural networks has attracted more and more attention.The pruning methods so far,such as the ones based on ADMM,lottery hypothesis,significance,and Auto ML,they often focus simply on the compression of model storage space,while ignoring the importance of the calculation workload for hardware deployment.To solve the problem that the compression rate of parameter remains several times that of calculation in the state-of-the-art mathods,this paper has studied and improved the neural network unstructured pruning algorithm by focuses on balancing the proportions of the amount of parameters and calculation in pruning.(1)Aiming at the problem that the previous pruning algorithm ignores the compression of calculation amount,this paper proposes a soft iterative pruning algorithm based on global statistical information.The distribution of parameter and calculation in the classic neural network is take into account,and the concept of calculation intensity is introduced.By comparing the calculation intensity of the network layer with the average calculation intensity of the entire network,the network layer is divided into a computationally significant layer and a parameterly significant layer.The global pruning algorithm is improved,according to the saliency scores of the weights of the two types of network layers,the pruning rates of the two types of network layers are set to perform independent pruning,balancing the pruning of network parameter and calculation.After setting the pruning rate of the two types of network layers,the algorithm can automatically obtain the hierarchical pruning rate,iteratively executes pruning and retraining,corrects incorrect pruning,and ensures the performance of the pruning network.(2)Based on Auto ML,this paper proposes a multi-objective constrained convolutional neural network unstructured pruning method based on genetic algorithm.This method models the multi-constrained convolutional neural network model pruning process as a multi-objective optimization problem during the unstructured pruning process of the deep convolutional neural network.By introducing an improved genetic algorithm,the best sparse network structure is searched in the convolutional neural network architecture space.On this structure,an optimal pruning result can be achieved by manually set the pruning target of desired memory usage and calculation workload.This method solves the problem above for the memory usage and the calculation workload in pruning are balanced.The experimental results show that the pruning method proposed in this paper can not only effectively reduce the parameter and calculation redundancy of the model,but also significantly reduce the calculation cost of the dedicated acceleration circuit,so that the sparse network after pruning is deployed to the embedded mobile platform and achieve accelerated computing.