Research On Structured Model Compression Algorithm In Deep Convolutional Neural Network

In recent years,thanks to the support of massive data and powerful computing resources,CNN(Convolutional Neural Network)has been widely concerned and researched by scholars,and has achieved excellent performance in many computer vision tasks.However,CNN’s parameter scale and reasoning time are also growing rapidly,which poses higher requirements for devices deploying high-performance CNN.Especially for resource-constrained embedded and mobile devices,this is undoubtedly a huge challenge.Therefore,it is of great significance for the development of deep learning to compress high-performance CNN to an appropriate scale so that they can run on resource-limited platforms.The structured model compression method can "downsize" the network model structurally,thereby achieving compression of the network scale and acceleration of the network reasoning.Under such background,the structural model compression of deep CNN is studied in this paper.The main contributions are as follows:(1)Deep Convolutional Neural Network Pruning Method Based on Canonical Correlation Analysis(CCA-Pr)is proposed.Most of the existing structured pruning methods rely on the criterion of "smaller-norm-less-important" and the layer-by-layer way to measure the importance of filters.The former may not be reliable in application,and the latter may fall into the local optimal solution.CCA-Pr uses CCA technology to learn the correlation between the filters’ corresponding features and sample labels,and combines the importance of lowdimensional features to determine the pruning of the filters.In addition,the global iterative pruning framework is adopted.Finally,the CCA-Pr method is validated by compressing the networks with different structures on the CIFAR-10 and Fashion-MNIST datasets.(2)Adaptive Pruning Method Based on Multi-perspective Pruning Criteria(MPC-APM)is proposed.This method first designs pruning criteria from two perspectives of filter and feature map,and then fuses them as a hybrid pruning criterion.In general,MPC-APM involves three pruning criteria,the first two considering filter information and feature map information respectively,the third considering filter information and feature map information simultaneously.The multi-perspective pruning criteria ensure the comprehensiveness and flexibility of pruning decision.On this basis,an adaptive pruning strategy is used to greedily select the criterion matching the state of the network model in each iteration pruning.The superiority of MPC-APM is verified by experiments on several network models on the CIFAR10 and CIFAR-100 datasets(3)Compression Method with Complete Filter Fusion(CFF-CM)is proposed.The fusion of current pruning methods requires complicated artificial design and cannot take full advantage of the relationship between all filters.CFF-CM further investigates and extends the fusion in the pruning method,focusing on fusing the filters information instead of measuring the importance/redundancy of the filters.The method fully fuses the filters in the current convolutional layer to narrow the width of the convolutional layer and achieves the same network topology as the filter pruning.In addition,CFF-CM is an extension of pruning.Finally,the experimental results prove that CFF-CM has excellent compression ability.For example,on the CIFAR-10 dataset,the VGG-19’s parameters and floating point operations are reduced by 92.12%and 67.24%respectively,but the accuracy is increased by 0.15 percentage points.