Research On Compression Method Of Deep Neural Networks

With the advent of the artificial intelligence era,a series of mobile terminals such as smart phones,drones,robots try to achieve intelligent tasks such as image recognition and object tracking through deep neural networks.Deploying deep neural networks on the mobile side has many advantages.However,deep neural networks are usually over-parameterized and there are serious redundancy in deep learning models,which leads to huge waste of computation and storage.At present,most of the depth models cannot be deployed on the mobile terminal due to their large size.To solve this problem,This paper presents a compression method of deep neural networks.The compression method of deep neural networks proposed in this paper can be divided into three steps.First of all,we use the network pruning strategy to prune the network after the normal training.This process turns the original network into a sparse network and achieves an initial compression of the network.Then through the K-Means++ clustering we get the clustering center of each layer.We use the clustering center value to represent the original weight value to realize weight sharing so as to reduce the number of weights and achieve deep neural network compression again.Finally,the weights of each layer are quantified through K-Means++ clustering,and the clustering center is retrained and updated.The quantization process reduces the number of bits used to represent the weights and achieves the final compression for the deep network.In this paper,experiments are performed on LeNet,AlexNet,and VGG-16 network based on the Caffe framework under Linux operating system.The compression method of deep neural networks proposed in this paper is simple and effective and eventually compress the deep neural network by 30 to 40 times.We also carry out detailed comparison experiments on pruning rate,clustering methods,optimization methods and other aspects.Experimental results show that with the compression method of deep neural networks proposed in this paper,deep neural networks achieve effective compression,which makes it possible to deploy deep networks on the mobile side.