Neural Network Performance Optimization Based On Streaming Distributed Architecture

As one of the core algorithms of artificial intelligence,neural network has higher and higher requirements for training speed,accuracy and architecture reusability.Currently training a medium-sized multi-layer neural network requires billions of computations.The limitations of traditional architecture such as low degree of parallelism,poor reusability,and weak scalability have been unable to meet the needs of neural networks in parallel computing.Therefore,this paper studies the neural network based on streaming distributed architecture to meet the needs of training speed improvement,architecture reuse and computing capacity expansion.This paper first proposes a neural network training model NN-S(Neural Network-Storm)based on Storm streaming distributed architecture.The data processing method is used to decompose batch neural network training tasks into multiple computing units for parallel execution.The parameters are updated synchronously after the batch data training is completed.In the Storm architecture,the Zookeeper network is used for distributed deployment of multiple servers.The NSMap method is used to convert the neural network algorithm into a computer identifiable topology,and the LeNet-5 and AlexNet network training is tested.The training results show that the NN-S model can significantly improve the speed of neural network training.At the same time,the NN-S architecture can recover quickly when node failure and network resource scheduling are abnormal,and has strong robustness.Based on the NN-S model,this paper optimizes the neural network traditional BP algorithm,combines the advantages of online updating BP algorithm and cumulative BP algorithm,and proposes the MixBP algorithm which is more suitable for Storm streaming architecture,which decomposes all training data.The computing units synchronize training and periodically update the neural network parameters.The test results of LeNet-5 network show that the neural network training method based on MixBP in NN-S model has better acceleration effect than online update BP algorithm and cumulative update BP algorithm.For the parallel optimization problem of Storm architecture,this paper analyzes the main factors affecting the degree of parallelism and calculation speed,compares the impact of thread allocation on network training speed,and tests the re-allocation of threads based on compute node load with LeNet-5 as an example.Significantly improve the speed of neural network training.In this paper,the distributed neural network training based on flow distribution is studied.The distributed neural network training model based on Storm and the optimized training algorithm are designed.It has reference significance for distributed neural network training.