Compression And Optimization Method For Deep Convolutional Neural Networks

Deep convolutional neural networks(CNN)are a fusion of Artificial Neural Networks and image processing based Computer Vision tasks.As a representative algorithm in Deep Learning,deep convolutional neural networks have achieved significant effectiveness compared with traditional methods in various visual recognition tasks since their birth.The performance of deep convolutional networks has been continuously improved in recent years due to the development of network models,expansion of the image datasets in various visual recognition tasks and the improvement of high-performance Graphics Processing Units(GPU)for parallel computing.The performance gradually catches up and even surpasses the recognition ability of trained human beings in some areas.The research on deep convolutional neural networks has become a hot topic widely concerned by researchers in Computer Vision,Machine Learning and Artificial Intelligence.Evolving to these days,the improvement of deep convolutional network model performance is increasingly dependent on the deepening of network layers and the accumulation of computing resources.As a result,the complexity of the network model is gradually increasing,and the parameter amount becomes larger and larger.Also the reliance on high-performance computing resources has become more serious.On the other hand,the application requirements for deep convolutional networks in various visual recognition fields are also increasing.There are many applications with limited storage and computing resources.The large amount of parameters and computational complexity of deep convolutional networks greatly limits their application in such scenarios,which has become an important flaw in deep convolutional networks.What is more,it is feasible to compress and optimize a deep convolutional network model from its principle.And it is also a further development direction of Deep Learning.The convolutional layer has the characteristics of parameter sharing and local perception.The parameters and the computation amount are reduced on the basis of the fully connected neural network,but this feature extraction method still contains a large amount of parameter redundancy and computational redundancy.This redundancy feature has been proven in many related work,also included in some structural features and training methods,which are designed to resist such problems as overfitting caused by the redundancy.Redundancy is an inherent feature of deep learning models.Eliminating redundancy and retaining task-related information has become the development direction of convolutional neural network models.As a starting point,focusing on the elimination of redundancy characteristics in convolutional networks,this paper proposes algorithm for streamlining the weight parameters and optimizing the operation of deep convolution networks from different angles,and trys to introduce Dictionary Learning into Deep Learning.The framework of an application scheme for running a deep convolutional network in visual recognition tasks with limited hardware resources is finally proposed.Specifically,the main works of this paper are summarized follows:1)Network compression method based on linear representation of convolutional kernels is proposedIn this paper,the correlation between kernel weights in deep convolutional neural network is analyzed firstly,it is considered that the correlation is the embodiment of network model redundancy.Therefore,a linear representation method is proposed,which uses a small number of template convolution kernels to represent all large number of convolution kernels to imply compression of the model parameter amount.The template extraction process is implemented by unsupervised clustering,and the representation coefficients of the convolution kernels are fine-tuned to reduce the precision loss after the representation process.This paper verifies the effectiveness of the proposed method through extensive experiments on different network structures.2)Network optimization method based on sparse reconstruction with undercomplete dictionaries is proposedIn the previous method,there is a shortcoming that only the parameters of the model are compressed without accelerating the computing.To solve this problem,this paper further proposes to introduce Dictionary Learning into convolutional neural network model,and eliminating the redundancy between weight parameters through the sparsity and orthogonality in Dictionary Learning.The strategy is to use an under-complete dictionary to sparsely reconstruct convolutional kernels to achieve parameter compression.The computing is also accelerated due to the exchanging of convolution operation and sparse reconstruction.This paper also proposes a strategy to redistribute parameters to adjust the original weight distribution before fine-tuning.This method introduces the characteristics of sparsity,orthogonality and model interpretability of Dictionary Learning into network compression optimization problem,and achieves the purpose of it.It has been verified in various visual recognition tasks and models.3)A vision recognition system on microcomputer based on network compression and optimization method is proposedBased on the above work,this paper implements an effective algorithm for optimizing the parameter and calculation amount of large convolutional networks.Therefore,this paper further considers how to apply the deep convolution network in a visual recognition system with limited hardware resources,and proposes to use the optimized convolutional network model to run on a microcomputer.This can realize a low power and computation resource consuming visual recognition system,with high precision and real-time operation.This program is verified in many such visual recognition tasks.The result greatly enhances the actual completion performance of visual recognition tasks and extends the application scenarios of deep convolutional networks.