Research On Model Compression Method For Convolutional Neural Network

With the enhancement of computer processing ability and the increase of the amount of data that can be trained,convolutional neural network model based on end-to-end training has become an important method to solve complex tasks in the fields of pattern recognition and computer vision,and has achieved remarkable achievements.However,a large number of parameters,complex model structure,and high memory demand of these high-performance deep network models limit their deployment on mobile devices such as mobile phones and wearable devices with limited computing capacity.Therefore,how to accelerate and compress the network model while maintaining the performance of the deep convolution network model has become a hot topic in the field of deep learning.Thesis studies the model acceleration and compression technology of convolutional neural networks and proposes some improvements and solutions to the existing problems and difficulties.The main research contents are as following:1.A compact convolutional neural network method based on local features is proposed.The traditional convolution neural network constructs global features by extracting local features from the whole sample to classify and recognize the samples,ignoring the local features of the sample image.In thesis,the local feature extraction layer is set in the network model to extract the local features of samples for training,so as to reduce the parameters of the convolution neural network model,and the probability synthesis layer is set to synthesize the multi-local feature recognition to improve the performance of the compact network model.The experimental results on the benchmark data set show that the method based on sample local features and probability synthesis can effectively improve the performance of the compact network model.2.A neural network model compression method based on depth feature map migration is proposed.Aiming at the problem that the traditional knowledge distillation method ignores the hidden layer knowledge of the network model,thesis takes the last layer of the teacher network and the student network as the reference layer and the guide layer respectively,and sets a small decoder after the reference layer and the guide layer by referring to the human visual mechanism,extracts the output feature map of the reference layer into a depth feature map and takes the feature map as the knowledge input layer Line passing.The experimental results on three benchmark datasets show that the model compression method based on depth feature map migration can effectively compress the network model parameters while ensuring the accuracy of the network model.3.A lane detection method based on quantitative attention distillation is proposed.In view of the problem that the high-performance lane detection model has high computational cost in complex scenarios,thesis uses the hidden layer generated attention map in the teacher network as knowledge and quantifies the attention map to help students learn through the network.Meanwhile,in the process of students' online learning,thesis adopts a teacher attenuation strategy to further improve the students' network optimization ability.In order to verify the effectiveness of the method,thesis conducts a comparative experiment on three-lane data sets,namely tusimple,culane,and bdd100k.The experimental results show that the lane detection method based on quantitative attention distillation has better detection performance.