Research On Parallelization Of Deep Convolutional Neural Network Based On Software Pipeline Technology

The data fitting ability of CNN is closely related to the depth of its model.Therefore,a deep network structure is usually selected to improve the representation of CNN for gaining better accuracy.However,the deeper the model is,the longer the inferring and training time would be.In this case,it has become a hot topic to speed up the inferring or training process of the model.In order to deal with the slow processing of deep network,this paper proposed a general method,called Pipe CNN,that parallelize CNN with the support of software pipeline.Experimental results show that this method can efficiently utilize devices in model parallelism,and widely improve the training speed of CNN.This paper introduced the software pipeline in the first place,and then studied three aspects of CNN for optimization,which were forward propagation,back propagation and loss function,respectively.The main work is as follows:(1)Firstly,in order to accelerate the forward propagation of CNN,this paper abstracted the forward process into working parts.All the working parts could be executed at the same time,with the support of communication by cyclic queue.Thus the forward propagation can run in a way of pipeline.(2)Secondly,in order to speed up the training process of CNN,this paper studied the data dependence in the back propagation,and then analyzed two gradient updating strategies in Pipe CNN to verify the feasibility of this method.(3)Thirdly,in order to improve the training accuracy of CNN,this paper delved into the focal loss function deeply,and then put forward the modification with the use of S-type function,which would make the model train better.