Research On Convolutional Neural Network Compression Method Based On Dynamic Pruning And Weight Resetting

Convolutional neural network has an important position in the field of computer vision,and its performance is gradually improved with continuous development and application.However,the number of layers,the amount of parameters,and the amount of calculations of the network have also increased significantly,which hinders the application of convolutional neural networks on mobile terminals such as smart phones and wearable smart devices with limited resources.In order to solve these problems,researchers have proposed many solutions to effectively identify and eliminate the unimportant parts of the convolutional neural network under the premise of minimizing the network performance loss,so as to achieve the purpose of compressing the model and accelerating the calculation.Network pruning is an effective way to compress and accelerate deep neural networks.It can significantly reduce the redundancy and complexity of the network.Nowadays,network pruning has gradually become a popular research,but there are still several issues worthy of more in-depth study.In view of two unsolved problems in network pruning,the analysis and research on convolutional neural network compression are carried out:(1)How to achieve a good compression effect with alternate training and dynamic pruning,without affecting the performance of the model after pruning;(2)How to better improve the performance of the compression model;carry out the analysis and research of convolutional neural network compression,the main work of this article has the following two aspects.(1)Firstly,this paper proposes an efficient pruning method based on dynamic thresholds.Existing pruning methods usually rely on pre-trained networks,and then perform a thorough pruning of all channels according to a certain pruning rate,and then fine-tune the pruned models.Although these methods can maintain good results in performance,the compression effect is not too ideal.The pruning method based on dynamic thresholds is to perform sparse training and pruning at the same time.After each epoch of training,all channels in the network are evaluated,and the unimportant channels with lower scores are removed from the model.The thresholds change with the iteration of training,and the scores of the channels also change,so during this process the pruned channel may be reconnected to the network.Multiple experiments of the model compression method based on dynamic pruning show that the pruned models can achieve better compression effects than existing methods.(2)Then,further reference the reseted weight to improve the performance of the compression models.Most pruning methods use the weights obtained from the full models training to fine-tune the pruned models.Research has found that the "important" weights learned from pre-trained models are usually useless for small pruned models,and the result of fine-tuning is usually not better than a model trained from scratch with randomly initialized weights.The ultimate goal of network pruning is to obtain the optimal structure,that is,channel numbers in each layer,not to inherit the weights of the complete models.The compression models performance improvement method based on reseted weights keeps the parameters or calculations of the pruned models and the complete model training process the same,calculates the epochs of pruned model training and performs training the pruned model from with randomly initialized weights.Experiments on multiple pruned models show that resetting the weights and training from scratch can achieve performance equivalent to or better than fine-tuning.