Research On Convolutional Neural Network Lightweight Method Based On Dilated Convolution And Piecewise Quantization

In recent years,with the rapid development of the mobile internet,mobile devices have been widely used by people.Convolutional neural networks have been the mainstream technology in the computer vision domain,and support various services for ubiquitous mobile devices.However,convolutional neural network models often have problems such as large scale,deep layers,high complexity,and high standard for hardware requirements,etc.To develop a large-scale convolutional neural network model which is more viable for edge devices of the mobile internet,the research on lightweight convolutional neural networks is increasing.How to make the convolutional neural network guarantee the model accuracy as well as reduces the parameter numbers of models and computational complexity to make the convolutional neural network model lighter gradually becomes a novel issue.Firstly,this thesis analyzes the domestic and international status quo of convolutional neural networks lightweight method concerning structural optimization design and model compression.Then,derived from the structural optimization design side,we do some research of convolutional neural network lightweight methods about dilated convolution,which is often used for multi-scale information fusion.Taking advantage of the characteristics of equal-interval sampling and expanding the receptive field of the dilated convolution,on the premise of ensuring the size of the receptive field and the feature map remain unchanged,this thesis proposes a lightweight method of convolutional neural network based on the dilated convolution.The method reduces the parameters of models and computational complexity efficiently,and it comprises the basic-type convolutional neural network lightweight method,the improved-type convolutional neural network lightweight method,and the fusion-type convolutional neural network lightweight method.Wherein,the basic-type convolutional neural network lightweight method uses one layer dilated convolution to approximate two layers ordinary convolution.It presents good lightweight performance.Based on the basic-type convolutional neural network lightweight method,the improved-type convolutional neural network lightweight method compresses the feature map by utilizing 1×1 pointwise convolution,which brings better lightweight performance with a drastic decay of network accuracy.The fusion-type convolutional neural network lightweight method combines the improved-type method with ordinary convolution,it guarantees the network accuracy by ordinary convolution as well as maintains the lightweight performance by the improved-type method,which effectively achieves good tradeoff between speed and accuracy.Secondly,derived from quantization at the model compression side,we use the piecewise method to improve the original powers-of-two quantization method and the additive powers-of-two quantization method,then this thesis proposes a lightweight method of convolutional neural network based on piecewise quantization.Including the piecewise additive powers-of-two quantization method and the piecewise powersof-two quantization method combined with the weight pruning idea.Among them,the piecewise additive powers-of-two quantization method is oriented to higher-bit quantization situations.It inherits the advantages of the additive powers-of-two quantization method to uniformly distribute the increasing quantization resolution.At the same time,it uses the piecewise method to make the base levels generated by the quantization function more suitable for the distribution of the original full-precision weights.Inspired by the idea that weight pruning can prun off the weights which have little impact on the network,a piecewise powers-of-two quantization method combined with weight pruning is proposed for lower-bit quantization situations.This method discards some small-impact weights in the quantization process,and makes low-bit quantization situations' limited quantization resources as much as possible to retain as much as possible the trend information for the weights of the big impact in the network.Finally,this thesis does some experiments on the lightweight method of convolutional neural networks based on the dilated convolution in the CIFAR10,CIFAR100,and KITTI datasets.Do some experiments on the lightweight method of convolutional neural network based on piecewise quantization in the CIFAR10 and CIFAR100 datasets.Then we combine the lightweight method of convolutional neural network based on dilated convolution and the lightweight method of convolutional neural network based on piecewise quantization,it realizes the lightweight of convolutional neural network from two sides of structural optimization design and model compression,and we do some experiments on it in the CIFAR10 dataset.The experiment results show that our method can achieve better lightweight effects.