Model Compression Based On Convolution Neural Networks

In recent years,the Convolution Neural Network(CNN)has achieved excellent results in multiple areas such as image recognition,speech recognition,and natural language processing.In order to further improve the accuracy of the networks,their structure is becoming more complex and their scale is becoming larger and larger,which makes traditional computing devices unable to effectively deploy these models.Highperformance devices are characterized by high cost and power intensive.The devices used in the industrial environment have strict constraints,and its price,computing power,network transmission bandwidth,memory,etc.are limited.In addition,there are strict requirements for the processing delay of the model,which urge the real-time processing speed of the model.This makes large and highprecision CNN unusable in industrial scenarios.Due to the redundancy of the CNN,we can use some effective model compression algorithms to compress the large CNN(reduce its parameter amount and computing amount),so that light-weight CNN can obtain prediction results accurately and efficiently in resource-constrained scenarios.Based on the CNN,we designed two model compression algorithms and we have applied them to a real-world application.The main contributions are summarized as follows:1)We propose a new lightweight CNN model:Super Interaction Neural Networks(SINet).First of all,in order to improve the information interaction of the model in width,we design Exchange Shortcut Connection,which can integrate information from different groups without any additional computing costs.In addition,in order to improve the information interaction of the deep network model in depth,we proposed Dense Funnel Layer and Attention Based Decision Layer.Finally,through a comprehensive comparison experiment on the Image Net dataset,we verify that SINet is superior to the state-of-the-art lightweight models and widely used network(such as VGG-16).Through the ablation experiments on the CIFAR-100 dataset,we confirmed the validity and applicability of the designed components in SINet.2)We propose a pruning algorithm that does not need to specify the pruning rules in advance and can learn the network structure during training,which is called Adaptive Pruning Algorithm(ADP).This algorithm divides the overall training process into two parts,first training the network weight parameters,then training the network structure parameters,and finally obtaining a lightweight bidirectional balanced network.ADP can transform a large model into a lightweight model during network training while maintaining the accuracy of the large model.Finally,we verified the effectiveness of ADP through experiments and demonstrated its advantages over related algorithms.We also combine ADP with different models,showing the universality of the ADP.3)Based on both of compressing methods mentioned above,we implement them in a real-world application.According to the characteristics of the face recognition algorithm,the network is compressed.While ensuring the accuracy of face recognition,the network computing and parameter amount are greatly reduced,and the network is accelerated,showing the great effect of the proposed compression method.Experiments show the effectiveness of two methods proposed in this paper.Compared with existing model compression methods,our approaches achieve higher compression rate and accuracy.Besides,our compressing methods also exhibit very flexibility and robustness in practice.