Research On Adaptive Method Of Filter Pruning In Deep Learning

Deep learning has achieved big success in the areas of image recognition,machine vision,natural language processing,etc.,solved many complex problems,and made a revolutionary breakthrough in artificial intelligence technology.However,complex depth models are increasingly becoming "expanding".Models are getting larger and more complex,and the requirements for computing power are getting higher and higher.How to effectively reduce the amount of parameters and reasoning time of complex models has become a key issue that artificial intelligence needs to solve urgently.In particular,the implementation and deployment of deep models in real-time applications such as online learning and incremental learning and edge artificial intelligence devices such as mobile phones and wearable devices.How to achieve deep model compression and acceleration on these devices has become the main goal of the model compression.At present,researchers have proposed a large number of model compression methods,such as pruning,knowledge distillation,model architecture design,etc.This article focuses on filter pruning,which is a structured pruning.Compared with unstructured pruning,it can run under a mature deep learning framework.However,most of the current work relies on pre-trained models,which require fine-tuning to restore reasonable accuracy,resulting in a large amount of retraining costs for the model.Therefore,in view of the poor adaptability of the current traditional filter pruning methods and the need for manual intervention,this paper deeply studies the redundancy of the depth model,and designs adaptive filter pruning algorithms from the perspective of correlation and information theory.The main research work is as follows:(1)Filter adaptive pruning method based on core center alignment.In order to alleviate the problem of the model over-relying on the pre-trained model during the pruning process,and reduce the retraining cost of the model.Different from the previous work,this article explains the filter behavior change rule during the training process from the perspective of the association relationship,and proposes a new filter pruning method by using the change rule.This method can prune filters with similar functions in the later stage of neural network training.According to this strategy,this article can not only achieve model compression without fine-tuning,but also find a novel perspective to explain the behavioral changes of filters during training.More importantly,the method in this paper has been proved to be effective on many CNN models.(2)Filter soft adaptive pruning method based on entropy and mutual information.First,by analyzing most documents,this article finds that most articles only weigh the importance of filters from one angle,such as analyzing the contribution of a single filter to the model,and the synergy between multiple filters.This paper integrates the two methods,proposes a paradigm that uses entropy and mutual information to evaluate the importance of filters,and analyzes its advantages over existing indicators.Secondly,this paper innovatively proposes a soft filter pruning rule based on regularization.This algorithm uses different filters to capture different features,so that the filter trimmed in each iteration is initialized orthogonally to the remaining filters when the next weight update is performed.This method effectively improves the generalization of the algorithm.Finally,the soft adaptive filter pruning algorithm based on entropy and mutual information proposed in this paper is tested on multiple data sets and multiple models,and the experimental results show the usefulness of the algorithm.In summary,this article proposes an adaptive filter pruning algorithm from two perspectives to address the low efficiency of the existing filter pruning algorithm and the need for manual intervention.And the performance of the proposed method is verified on the existing public data set.The research in this paper provides a new perspective and method for the field of pruning,and has certain theoretical and application value in the field of model compression.