Research On Compression Method For Convolutional Neural Network Based On Pruning

Nowadays,the Convolutional Neural Networks(CNNs)have achieved impressive performance on many computer vision related tasks such as object detection and image recognition.However,the remarkable performance of CNNs has been accompanied by a significant increase of computation and memory costs,which prohibits their usage on resource-limited environments such as mobile or embedded devices.To this end,the CNN compression and acceleration has recently become an emerging area where researchers are devoted to reducing the consumption of computation and memory.Pruning is one of the predominant approaches for CNN compression,which can effectively find out and remove the redundant part of CNNs with minimal loss.In this thesis,two structured pruning methods and are proposed to compress and accelerate modern CNNs.The contributions are summarized as follows:1.Existing unstructured pruning methods lack the practical implementations to achieve the realistic acceleration.To address the above-mentioned problem,this thesis proposes a structured Saliency-based Global Filter Pruning(SGFP)approach.The SGFP calculates the gradient information of filters generated by multi-batch samples during training process.Specifically,the normalized magnitude of the gradient value is defined as the saliency criterion,which measures the effect of removing filters on network performance.The approach can construct a global ranking by setting a global pruning rate.With global ranking,one-shot filter pruning can be done effectively.In addition,SGFP adopts the group pruning to solve the misalignment problem when pruning the filters of Res Net.Empirical results demonstrate that the proposed approach can better trade-off accuracy for FLOPs.2.To eliminate the need for the expensive prune-retain cycles,this thesis proposes a filter pruning approach by combining the momentum with SGFP.The approach contains four steps.Firstly,it takes the mean of the momentum magnitude that belongs to all nonzero filters for each layer.The resulting proportion is momentum magnitude contribution for each layer.Then,it uses saliency criterion in SGFP to evaluate the importance of each filter and uses a pruning rate to select unimportant filters for each layer.Next,it sets the value of selected filters to zero,which can temporarily eliminate their contribution to the network output.Particularly,the pruning rate is decayed.Finally,it regrows filters proportionately by enabling the zero-valued(missing)filters which have the largest momentum magnitude.The extensive experiments show that the proposed approach can improve the pruning accuracy,especially for large CNNs such as Res Net56.