Research On Convolutional Neural Network For Compression Algorithm

With the development of deep learning,convolutional neural networks are applied to various tasks of computer vision,such as object classification,detection and segmentation.However,due to the large volume and high computational complexity of the convolutional neural network model itself,it hinders its deployment to terminal devices such as embedded devices with limited memory and power consumption.At present,many model compression methods based on pruning and quantization have emerged.They have achieved remarkable results in reducing the size,parameter number and calculation of the model.But most of these compressed models require special hardware and software support,which increases deployment costs.In addition,these methods are mainly used in classification tasks,and rarely directly used in detection tasks.In order to reduce the deployment cost of the compression model and improve the efficiency and scope of use of model compression,this paper introduces a three-stage model compression method: sparse training,group pruning,and knowledge distillation,so that the compressed model does not need the support of special hardware and software.The main research contents of this paper are as follows:1.For the object detection network,the thesis proposes a new structured channel pruning method,called packet channel pruning.Firstly,the network is sparsely trained,and the unimportant channels are selected.Secondly,the different layers in the target detection network are divided into multiple groups according to the functional modules,and each group is given different pruning ratios.Then the pruning thresholds of each group were obtained according to different pruning ratios of each group;Finally,perform channel pruning on the feature layer in the current group according to the pruning threshold in each group2.In order to reduce the accuracy loss caused by high pruning rate,knowledge distillation is introduced.In this thesis,the unpruned network is used as the teacher network,and the pruned network is used as the student network.In particular,the spatial attention map is extracted from the feature layer of each group.The spatial attention map is used as the main knowledge of distillation,and the prediction of the class and the prediction of the object position in the teacher network are used as auxiliary knowledge.In the experiment,YOLOV4 is used as the target detection network,and PASCAL VOC is used as the training data set.The proposed method reduces the parameter amount of the model by 64.7 % and the calculation amount by 34.9 %.At the same time,compared with the original network with 256 M size and 87.1 accuracy,the compressed network model in this paper reaches 86.6 accuracy with 90 M size.