The Research On Optimization Of Convolutional Neural Networks Inference With Resource-Constrained Settings

In recent years,advances have shown that Artificial Intelligence(AI)performs excellent in the tasks of image classification,autonomous vehicle and speech recognition with the rapid development of computer technology.The new generation of artificial intelligence technology,e.g.,deep learning is gradually infiltrating into more and more fields and promoting social development.As one of the most popular used deep learning model,Convolutional Neural Networks(CNNs)achieve good results in the tasks of image classification and face recognition relying on local perception,weight sharing,and temporal or spatial subsampling.CNNs perform excellent when the size of datasets is sufficiently large.Nevertheless,when public datasets are not big enough for training the model for new application scenarios,it is painful to obtain sufficient training examples,especially when the samples have to be labelled manually.Besides,training and inference using CNNs requires significant resources of energy,computation and memory usage.Therefore implanting deep CNN models trained and executed on high performance GPU clusters to resource constrained devices,i.e.,internet of things(IoT)devices,which have permeated into every aspect of modern life,is not appropriate and impractical.Compression technology is an important and popularly used tool to accelerate the training and inference of CNN models.This thesis proposes a CNN acceleration algorithm based on compression technology.The main research contents and innovations are as follows:This thesis proposes a new compressed CNN model termed as CS-CNN for image classification by modifying the input layer while the width of the network(the number of nodes of each layer)is reduced.CS-CNN incorporates the theory of image compression at the input layer of CNN models to reduce the dimensionality of the input layer.The projection matrix is generated by the most important information of the input which is extracted via singular value decomposition.In this paper,the convolution function in the TensorFlow framework is exploited to accelerate the compression stage.CS-CNN adds a convolution layer with fixed weights between the input layer and the first convolutional layer to compress the original data and extract useful information.The parameters of the new convolutional layer are fixed and are not updated during training.At the same time,the number of parameters of the input layer dominates the total number of parameters in the entire network,so the significant compression of the input layer can avoid overfitting when the samples are insufficient.This is an optimized CNN framework that can be applied to many embedded vision systems and a large number of IoTs devices.Finally,the proposed framework CS-CNN is verified by experiments.In this thesis,the experiments are conducted on the open datasets MNIST and CIFAR-10 which are popularly used to evaluate deep learning tasks.The results demonstrate that CS-CNN can greatly speed up the training and inference process through multiple performances.At the same time,when the dataset used for training is small,CS-CNN has higher classification accuracy than the traditional CNN model.