The Research On Optimization Of Convolution Neural Network Parallel Algorithm Based On Distributed Environment

With the increase of depth learning network scale,the number of network training parameters increase rapidly and the network training time increases.The convolution neural network is a part of deep learning.Although the number of parameters is reduced by the use of weight sharing,the number of parameters is still large and the time-consuming problem of network training is still too long.Existence.In order to solve this problem,the convolution neural network in distributed environment is realized,and the asynchronous parameter updating algorithm(B-APU)based on binary tree and the time factor based FIFO scheduling strategy(FIFO-T)are proposed to optimize the distributed convolution neural network.The main work is as follows:(1)We realized distributed convolution neural network based on Spark network cluster under multi machine environment.Through the integration with Spark platform,we explore a parallel scheme of convolution neural network in multi machine environment.We realized distributed convolution neural network based on Spark platform.(2)We designed B-APU(Asynchronous Parameters Update Based Binary-tree)algorithm to solve the problem that the distributed convolution neural network has a cluster node waiting for synchronous data parallel in training.The B-APU algorithm combined the binary tree with synchronous and asynchronous parallel methods.First,the network nodes were constructed according to the processing speed of the computer.The difference between the values of the left and right nodes of the same subtree is small and the left subtree node is the fastest node in the binary tree,and the upper layer of the two tree is the calculation speed phase,and the lower layer is the faster node.Secondly,for the parameter updating,the nodes of the same layer used the parameter averaging method to ensure the accuracy.The asynchronous random gradient descent algorithm is used between different layers to improve the training speed,and the transmission of each layer is carried out according to the depth of the binary tree to ensure that the fast speed nodes have priority to transfer the parameters.The node communication with faster nodes is is in parallel with the calculation of slower nodes,thus reducing network communication time in network training.(3)We designed the FIFO-T(First In First Out Based Time)operation scheduling strategy for the FIFO scheduling strategy of the distributed convolution neural network,under the Spark platform,which simply considers the node waiting problem brought by the job submission order.In view of the difference in job execution time that causes nodes to wait for each other,the FIFO-T job scheduling strategy uses the job execution time as one of the factors to judge the priority of the job scheduling,thus shortening the time that the cluster nodes wait for each other because of the inconsistency of the execution time.The experimental results show that the B-APU algorithm and FIFO-T scheduling strategy are applying to the training of distributed convolution neural network achieves better results.