Research And Application Of Compression Algorithm For Deep Neural Network

In recent years,neural network algorithms have shown outstanding performance in object recognition,object detection and semantic segmentation.In general,however,a network with good performance has a very large number of parameters and requires a lot of memory and computing costs.Therefore,considering the limitation of hardware resources on mobile devices,how to apply high-performance networks in real life is still a problem.In order to solve this problem,neural network compression technology has become a new research direction.The goal of network compression is to analyze the redundancy of deep neural network,reduce the size of network storage and speed up network computing.Among various compression techniques,network sparsity is an important method which removes parameters from the model.Therefore,in this paper,the author studies the sparsity strategy in neural network compression,including face recognition network and its compression & acceleration,a new neural network compression algorithm based on attention mechanism,and technical standardization of neural network compression.The innovations and contributions of this paper are summarized as follows:1.Mobilization and application of face recognition model.In this paper,face recognition technology is utilized to train a high-performance network on large face dataset and special face dataset.Then,according to the characteristics of the face recognition task and the network structure of model,we compress and accelerate the network to make it run on the embedded device at a high speed.2.Self-attention mechanism of generative adversarial networks is usually utilized to find relevance and correlation in features.Combining self-attention mechanism with network sparsity technology,this paper proposes a new network compression algorithm,which achieves good results in benchmark datasets.3.The author participated in a group of making national standard for neural network representation and compression.During conferences,the author proposed several technical proposals,including the technical framework and compression algorithms.Most of these proposals were received by the working group to promote the standardization of neural network compression.This section mainly introduces the author's contribution to this standard.