Parallel Algorithm Design Based On Heterogeneous Computing Platform For Neural Network Training

Nowdays,Artificial Intelligence(AI)can be seen everywhere in human's life.Many industries have achieved tremendous growth through artificial intelligence.The most important technology of AI is Neural Network,and the extensive use of AI cannot come without the great advances made in Neural Network technology.However,the further step of development in Neural Network technology still faces many difficulties and challenges.Recently,one of the challenge Neural Network faced is training.The essence of training is an optimization process based on numerous training data iterating incrementally.The process need great computing power and efficient optimal solution search method.To solve the problem faced by Neural Network training process,this paper makes exploration and analysis.Based on the great computing power of heterogeneous computing platforms,some parallel optimization algorithms have been designed and implementation with OpenCL.Firstly,a parallel BFGS Quasi-Newton algorithm is implemented to accelerate Neural Network training process;secondly,to enhance the global exploratory capablity of Neural Network,a multi-swarm parallel PSO algorithm is proposed in this paper;thirdly,BFGS-PSO hybrid algorithm is designed to obtain a higher convergence rate in Neural Network training process.The experimental results show that,compared to the traditional PSO algorithm implemented on CPU,the multi-swarm parallel PSO algorithm gets 35 times acceleration with a smaller error,while the parallel BFGS Quasi Newton algorithm has achieved 430 speed up.Further more,a good convergence rate is shown by BFGS-PSO hybrid algorithm,compared to BFGS Quasi Newton algorithm,the convergence rate has been increased by 5.5 times,and with the same execute time,BFGS-PSO hybrid algorithm has the smallest training error 1.12% among three algorithms implemented in this paper.