Research On Compression Algorithm And Application Of Deep Convolution Neural Networks Based On Network Pruning

Deep convolutional neural networks have made a big success in recent years and become one of the most popular research directions.Its success has been accompanied by the number of parameters increasing exponentially,resulting in expensive computational and storage costs.To address this problem,model compression has emerged.This thesis mainly studies model compression methods based on network pruning.The main work is divided into three parts as follows:(1)This thesis proposes a filter pruning algorithm based on similarity clustering.Unlike previous filter importance-based pruning methods,this algorithm focuses on the similarity of filters in the same layer and removes the filters with high similarity.In this algorithm,the similarity is measured using Euclidean distance,and the smaller the distance is,the more similar the filters are.The similar filter pair with the smallest distance between filters is selected,and the k-nearest neighbor distance sum is calculated for each filter in the filter pair,then the filter with the smallest distance sum is pruned.The removed filter can be replaced using its nearest neighbor filter.In this thesis,we conduct experiments on several datasets to verify the effectiveness of this algorithm.On the CIFAR10 dataset,our method reduces more than 70%of FL OPs and parameters on GoogLeNet,and the accuracy is even improved by 0.09%over the benchmark model.(2)This thesis proposes a filter pruning algorithm based on layer redundancy.Most of the previous works use global uniform pruning rate.We believe that the global uniform pruning rate cannot obtain the optimal pruning effect,so we propose a filter pruning algorithm based on layer redundancy.The algorithm uses the Euclidean distance to measure the similarity between filters and the Taylor expansion loss function to approximate the pruning sensitivity of filters,and then combines them to measure the redundancy of each layer in the convolutional neural network.Depending on the redundancy,different pruning rates are set.Through experimental validation,this method performs well on most different network models.Among them,after removing 55.1%of FLOPs from ResNet-56 on the CIFAR10 dataset,the accuracy reaches 93.27%,which is 0.13%higher than the accuracy of the filter pruning method using similarity-based clustering only.(3)This thesis designs and implements an application system based on network pruning.The system is developed by using PyQt5 framework with the pruning algorithm proposed in this thesis.The system includes the functional modules of model training,model pruning and image recognition.Users can choose models for training and pruning.By using this system to compare the changes of the model before and after pruning,we can demonstrate the effectiveness and practicality of the network pruning algorithm proposed in this thesis.