| Model compression is a practical and widely used technique in deep learning,which can achieve lightweight model by compressing the structure of neural network model and its data accuracy,so network pruning and network quantization become the most common model compression techniques.Proper use of model compression techniques can shorten the forward inference time of models,reduce the back-end hardware pressure,and facilitate AI projects.Therefore,the following studies are conducted on network pruning and network quantization techniques for generic neural network models.(1)A frequency response-based convolutional neural network pruning method is proposed.Where the influence of different frequencies of information in the input data of neural network on the network accuracy and the influence on the similarity between objects is studied,and based on the experimental conclusion that the low-frequency information in the input data is relatively more important,the frequency response-based convolutional neural network pruning method is proposed,and finally the feasibility of model pruning from the perspective of frequency domain is verified through comparison experiments and ablation experiments.(2)A neural network pruning method based on low-frequency information quantity.Based on the study of(1),the role of frequency information in the network pruning work is further investigated.Through experiments,it is found that among the many channels of the neural network,the channels corresponding to feature maps with a higher proportion of low-frequency information are more important for the neural network.Based on this conclusion,a network pruning method is proposed,where the amount of low-frequency information and the average rank of channels are used as the basis for slimming the neural network.(3)Knowledge distillation based assisted convolutional neural network quantization method.In this chapter,the advantages and shortcomings of existing posttraining quantization and quantization-aware training are analyzed,and a neural network quantization method is proposed by combining knowledge distillation,a model compression method with knowledge migration characteristics.And the advantages of this method over the existing post-training quantization methods are verified by experiments.Finally,the network pruning and network quantization methods are integrated to perform a comprehensive model compression of neural networks,achieving a complete process of neural network deployment.In brief,two network pruning methods are proposed from the frequency domain perspective,starting from the input data and the characteristics regarding frequency in the feature maps.For further model compression,a network quantization method is also proposed by drawing on the idea of knowledge distillation.Through network pruning and network quantization,and using a deep learning compiler,a comprehensive model compression is accomplished and a true sense of neural network deployment is achieved. |
