Compression Method Research Of Deep Convolution Neural Network Model By Low-rank And Sparse Decomposition

In recent years,the highly efficient convolutional neural network model lead to exciting results and widely utilized in machine vision detection.However,the existing deep convolutional neural network model requring high memory consumption,large amount of calculation and numerous parameters,which hinder its deployment on resource-limited platform.Therefore,how to apply the deep model to resource-limited platforms while maintaining its performance has become a key issue.Model compression can effectively solve the above problems.It can reduce model redundancy,storage space and energy consumption by pruning redundant connections and channels or decomposing convolution kernel.The filter of the deep convolutional neural network tend to be low-rank and sparse.However,existing work often ignores this feature,resulting in the compression process being susceptible to iterative training,low compression ratio and reduced accuracy.Aiming at this problem,this thesis studies the compression method of deep convolutional neural networks,and proposes a model compression method that decompose the weight matrix into low-rank components and sparse components.Its main research content are as follows:(1)Propose a unified framework integrating the low-rank and sparse decomposition of weight matrices with the dynamic analysis of weight importance.Firstly,the GoDec algorithm decomposes the weight matrix into a low-rank component matrix and a sparse component matrix,and further decomposes the low-rank matrix.Iteratively optimizes the individual components to obtain a global optimal solution.Then,before each pruning,analyze the importance of the weights of each layer and pruning them in order of importance.Finally,the network layer without low-rank components is used as a special case of decomposition.(2)The distribution of elements in the sparse component matrix obtained by the GoDec algorithm decomposition is irregular.Aiming at this problem,this thesis proposes a compression method of regular sparse and low-rank decomposition.A regular sparse-inducing norm is introduced in the sparse component.The norm uses the correlation between elements of the sparse matrix to constrain the matrix to a regular sparse matrix with structured of nonzero elements.It can improve the sparsity of the model,reduce the number of indexes and then reduce the storage space.(3)In the experimental analysis stage,we first analyze the compression sensitivity of the model network layer,and determine the compression ratio of each network layer by the sensitivity.Finally,on the CIFAR-10 and CIFAR-100 datasets,we performed compression experiments on trained AlexNet,VGG-16 and GoogLeNet models.Experimental results show that the two methods proposed in this thesis can significantly reduce the parameters for the convolutional and the fully connected layer of each model.In the case of no significant decrease in classification accuracy,the compression ratio of AlexNet was 10 times,and the compression ratio of VGG-16 was as high as 16 times.Compared with the "three-step pruning" method and the Tucker decomposition,the proposed method can obtain better classification accuracy under the same compression ratio,and the optimized compression method performs better.