Research And Application Of Deep Model Compression Technology

In recent years,Deep Neural Network has continuously refreshed the best performance in tasks such as computer vision and natural language processing,and becomes the research direction that has attracted the most attention.However,the parameters of the deep model are huge,and the storage cost and calculation cost are too high,which seriously limits its deployment and application on embedded devices and mobile devices.From the perspective of model compression,the task of pruning and optimizing deep neural networks is divided into two parts in this paper: 1)model pruning technology for a single convolutional layer,2)exploring the optimal compression combination of each layer of the model.Based on the research,this paper designs and implements a universal model compression tool,so that more deep learning users can quickly optimize and deploy their model.First of all,for the pruning strategy of a single convolutional layer,this article will first explain the principle of compression algorithm under weight granularity,kernel granularity and channel granularity from the perspective of weight contribution,and analyze the unreasonableness of current filter pruning method based on weight contribution.To this end,this paper proposes two novel strategies for filter pruning.Starting from the completeness of the feature space extracted by the filter,we use the orthogonality between the filters as the pruning criterion.Starting from the information richness in the output feature maps of different filters,we use feature rank as the pruning criterion of the filters.Secondly,aiming at the combination of pruning proportion in each layer of the model,this paper proposes to combine reinforcement learning to realize the automatic compression of the whole model.Specifically,the structural attributes,pruning ratio,and final compression performance of each layer of the model are mapped to the states,actions,and rewards in the reinforcement learning algorithm.For the characteristic of continuity in the pruning ratio value range,the RNN controller and DDPG based algorithms are proposed.Next,the paper designs and implements the Pliers model compression tool.This tool integrates different pruning granularity and automatic compression algorithms mentioned before.Users only need to upload and manage datasets and models through the front-end interface,and then set the training hyper-parameters and start the model compression engine through the configuration interface.The compression tool will automatically compress the model according to the parameter information in the configuration,and present the final comparison result of the compression performance to the user in the form of a report for the user to download and use.Finally,this paper performs experimental verification and system testing on the proposed algorithm and compression tool,respectively.During algorithm verification,the Vgg16 model trained on the CIFAR-10 dataset is subjected to pruning comparisons with different granularities and automated compression experiments.In the end,the parameters of the model can be reduced by 85%,the calculation amount is reduced by 80%,the speed is doubled,and the accuracy loss of the model is within 1.5%.In the system test section,we first introduce the deployment environment of the compression tool,and then the functional tests are performed on the configuration module,data management module,and task management module to verify the availability of the system.