Research On Algorithms Of Compressing Convolutional Neural Networks Based On Deep Compression

In recent years,deep learning has been a hot topic in the field of artificial intelligence.As a classical learning model of deep learning,convolutional neural network(CNN)has achieved a series of excellent results in image classification,target detection,natural language processing and other fields.Convolutional neural network model has the characteristics of large number of parameters and high computation.It is this complex multi-level network structure that makes the model have high accuracy of prediction.Because of the popularity of intelligent devices,the demand for convolutional neural networks to be transferred to embedded devices is increasing and a lot of neural network compression algorithms emerge at this historic moment.Among them,"Deep Compression" algorithm is widely used in large-scale neural network compression because of its outstanding compression performance.In this paper,we focus on the convolutional neural network compression algorithm and make some improvements based on "Deep Compression" algorithm.The main contents are as follows:(1)This paper proposes an algorithm of dynamic pruning and dynamic quantization based on "Deep Compression" algorithm.The "Deep Compression" algorithm has remarkable compression effect on compressing large convolutional neural networks.However,it ignores the different orders of magnitude of weights and contribution to output of different layers in the stage of pruning and quantization.To solve this problem,this paper proposes an algorithm of dynamic pruning and dynamic quantization,which classify different levels of weights,and set different thresholds and clustering numbers respectively.When setting threshold,we change the simple specified value to a proportional threshold.The experimental results show that the compression performance is better than the traditional Deep Compression algorithm.(2)This paper proposes an improved "Deep Compression" algorithm for compressing complex convolution neural networks.At present,much research has focused on real convolution neural network(Real CNN).As far as we know,there is no paper discussing compression algorithm for complex convolution neural network(Complex CNN).Compared with Real CNN,Complex CNN is easier to optimize.Complex CNN has better generalization ability and learning potential.In addition,it is more robust to noise..In this paper,we extend the popular Deep Compression algorithm from the real field to the complex field,and propose an improved deep compression algorithm for compressing complex convolution neural networks.Because of the complex form of the weight in complex convolution neural network,considering the correlation between imaginary part and real part of the complex weight,we made the following improvements in pruning,quantization and Huffman coding: 1)pruning stage: using modulus of complex number to compare with threshold value;2)quantization stage: using two-dimensional K-means clustering algorithm to cluster complex weight and the clustering centroid is a shared weight;3)Huffman coding: Because the correlation between the imaginary part and the real part of complex weight does not affect the result of Huffman coding,the real part and the imaginary part of weight are coded separately.The proposed algorithm can compress the network on CIFAR-10 dataset eight times,and the accuracy loss is less than 3% without retraining.On the IMAGENET 2012 dataset,our method achieves 16 times when the compressing the model,and the accuracy loss is less than 2% without retraining.