Research On Lightweight Convolution Neural Network Model For On-Device Learning

As one of the representatives of deep learning model,convolutional neural network model has been widely used in the fields of auxiliary medical diagnosis,automatic driving and ‘Smile to Pay',which greatly facilitates people's lives.The lightweight technology of convolutional neural network is the basic research to reduce the model cost by studying the parameters and structure of the model,which can effectively improve the application potential of the model.This thesis studies the optimization method of convolution operation from different aspects,starting from the filter characteristics and network structure of convolution neural network model.Firstly,the filter in convolution kernel is analyzed statistically.Considering the numerical and spatial characteristics,a method pruning parallel from two aspects is proposed.Then,the feature expansion convolution module is designed by using the idea of feature reuse.Finally,the lightweight model is designed by combining the two methods,and the verification is carried out in the refuse classification.The main work of this thesis is as follows:Firstly,a method pruning parallel from two aspects is proposed which can be used to optimize convolution operation.Through statistical analysis of the filter in convolution kernel,it is found that the most of the L2 values of the filter are concentrated in a relatively small range,which will make it difficult to determine the appropriate threshold value when pruning.Therefore,the clustering analysis of filter in convolution kernel channel is carried out by using agglomerative hierarchical clustering algorithm.The filter is divided into redundant clusters with similarity in spatial distribution and discrete clusters by using parameter to control clustering process.For these two clusters,the filter in redundant cluster is cut from the perspective of spatial similarity,the filter in discrete cluster is cut from numerical point,and the number of parameters in convolution operation is reduced by using the method of parallel pruning from two aspects.The method takes into account the characteristics of the filter in space and value,can effectively reduce the number of parameters of the model,optimize convolution operation,and successfully apply it to VGG16 model.The results are good in the experimental verification of CIFAR10 and MNIST data sets.Then,a feature expansion convolution module based on feature reuse is proposed.From the view of feature reuse,the feature expansion convolution module is designed by introducing the cheap operation idea in ghost module.In this module,the output channel number of the standard convolution module is reduced and the multi branch structure is introduced.The output characteristic graph of standard convolution operation is transformed and fused by the cheap operation on each branch to generate new feature map.The final output of the module is obtained by combining the feature graphs generated on each branch.The feature expansion convolution module reuses the features of the model with the idea of cheap operation,which enriches the hidden information of the feature graph while reducing the calculation amount of the model,and improves the performance of the model.The feature expansion convolution module is replaced by the standard convolution module in VGG16,and the lightweight VGG16 model is designed.The classification results are good on CIFAR and Image Net ILSVRC2012.Finally,based on the feature expansion convolution module,a lightweight classification model which can run on the embedded device is designed for the garbage classification task.And the two angle parallel pruning method is used to re optimize the model.It verifies that the two methods proposed in this thesis can optimize the model structure from different angles at the same time.There are 26 figures,13 tables and 94 references in this thesis.