Research On Deep Neural Network Compression Method Based On Soft Masks Learning

With the development of deep learning,neural network models in face recognition,natural language processing,medical image processing,and other fields have excellent performance.However,with the emergence of more and more complicated tasks,the size of the neural network is also intensified.The complexity of the neural network structure and the rapid increase in the number of parameters require a large amount of computing resources and memory resources in actual deployment,which greatly limits the application scenarios of the network model.Therefore,complex network models face the dilemma of being difficult to deploy on edge devices with limited computing resources.The channel pruning algorithm is one of the most effective model compression techniques to solve the above problems.In this thesis,two new channel pruning algorithms are proposed,and the specific research contents are as follows:(1)The current static pruning works ignore the influence of instance complexity on the importance of filters,so this thesis proposes a pruning algorithm based on instance complexity weighted soft masks.The algorithm assigns different weights to input instances and calculates the weighted sum of soft masks to simulate the influence of instances with different complexity on the filter.In addition,manual selection of thresholds is still required after sparsening soft masks with existing sparse regularizers,which often results in large accuracy loss for pruned networks.Based on this problem,a new sparse regularizer is proposed in this thesis that can effectively polarize the soft masks into two parts near 0 and 1,thus effectively reducing the accuracy loss after pruning.The algorithm has great performance on different datasets and network models.(2)Since the current pruning algorithm usually adopts the method of one-level optimization to optimize the parameters or mask,this optimization method is easy to cause the parameters to overfit in the training data or fall into the local optimal solution in the optimization process.In addition,the pruning algorithm relies too much on pre-trained network,which can lead to additional computational overhead.Based on the above problems,this thesis proposes a pruning algorithm based on differentiable architecture automatic search.In the process of pruning,the algorithm only needs to call the pre-trained network once to calculate the information entropy of the feature graph generated by the filter and act as the initial soft mask.In addition,the algorithm jointly optimizes the soft mask and the network parameters by the way of Bi-level optimization so as to realize the differentiable search of the network structure.Experimental results on various data sets and network models demonstrate the effectiveness of the proposed algorithm.(3)Design and develop an Android-based mobile image recognition system,which is based on the NCNN framework and realizes efficient and accurate image recognition by deploying a compressed network model.The system realizes the functions of image selection,image recognition,and output key information.