Research On Optimization Of Convolutional Neural Network Compression Algorithm

With the rapid development of the information age,all kinds of intelligent terminals have entered thousands of households,such as communication devices,service robots,intelligent bracelets,etc.These devices generate a large amount of and complex data all the time,leading to low efficiency in data processing by convolutional neural network.Therefore,a better convolutional neural network model is needed to process data in real time.These neural network models should have good performance both in processing speed and accuracy.In this context,researchers have studied many convolutional neural network compression algorithms.Different methods have been used to make the model simpler.However,the convolutional neural network model can be compressed to improve data processing efficiency under the condition of no loss of accuracy or less loss of accuracy.Pruning and quantization algorithms are two of the compression algorithms.As one of the key parameters of pruning,the sparsity rate determines the sparse effect after pruning,and is closely related to the complexity,accuracy and application of convolutional neural network.This paper is aimed at pruning and quantization algorithm to complete the following two aspects of work:1.A snip prepruning optimization algorithm based on dynamic sparsity rate(PDSR)is proposed.In the process of convolutional neural network model training,the sparsity rate is often artificially set according to experience,which cannot be changed dynamically with the change of experimental environment and data,and the accuracy of experimental results is difficult to determine.Therefore,it is necessary to set different sparsity rates for multiple experiments to get a better value.To solve the above problems,this paper introduces the dynamic sparse rate,optimizes the snip prepruning algorithm,and proposes the PDSR algorithm.The algorithm dynamically calculates the sparsity rate through the connection sensitivity of weights,and judges whether the weights are connected in the convolutional neural network according to the dynamic sparsity rate,and then prepruning the convolutional neural network.The proposed algorithm solves the problem of the uncertainty of the empirically specified sparsity rate and the need to change it manually in the snip algorithm.The experimental results on various convolutional neural networks show that the PDSR algorithm is superior.2.A hybrid pruning-quantization compression algorithm based on principal component analysis(PCAHC)is proposed.Among the existing pruning algorithms,some scholars proposed to use the scaling factor?of the BN layer to measure the importance of channels.However,However,the scaling factor has a small numerical difference of each layer,and even pruning by 50%results in a significant decrease in the algorithm accuracy.Therefore,this paper proposes PCAHC algorithm to solve the above problems.This algorithm discretizes the scaling factor by calculating the square of the scaling factor,and measures the importance of each channel,theoretically highlighting the importance of some channels.In addition,pruning is carried out according to the principle of principal component analysis.Finally,the pruning model is quantified.By changing?into?~2,the algorithm correspondingly changes the penalty function imposed by the original algorithm.The simulation results show that the PCAHC algorithm is better.