Research On The Convolutional Neural Networks Optimization Based On Cooperative Particle Swarm Algorithm

In recent years,convolutional neural networks,as an important tool in the field of deep learning,have been applied to many computer vision tasks with excellent performance.But the training of convolutional neural networks is still a challenge.The training problem of convolutional neural networks is essentially a large-scale optimization problem.When the dimension of the problem increases,many heuristic algorithms lose their performance on low dimensional problems.By using multiple modules to search the problem subspace at the same time with information exchanging between module,the cooperative strategy can improve the ability of the algorithm in solving highdimensional problems.Firstly,the basic principle and network structure of convolutional neural networks are introduced.Then introduced the principle and algorithm flow of basic particle swarm optimization(PSO)algorithm.The design idea of cooperative particle swarm optimization(CPSO)algorithm for PSO algorithm is analyzed,and their advantages and disadvantages of several different CPSO algorithms are discussed.On this basis,a hierarchical CSPO(Hierarchy Cooperative Particle Swarm Optimization,HCPSO)algorithm is proposed.By combining several CPSO algorithms with different packet sizes in a hierarchical and progressive manner and making full use of the characteristics of the CPSO algorithm with different packet sizes,the algorithm has the ability to jump out of the local optimal value while keeping the convergence speed.The results of comparison on several complex test functions,show that the hierarchical CPSO algorithm has better performance than CPSO algorithm and PSO algorithm.In the end,we use CPSO algorithm to train LeNet-7 network on CIFAR-10 data sets,and compare the performance of various training methods.The results show that the CPSO algorithm has obvious disadvantages compared with the gradient descent algorithm,but it is found that the performance of HCPSO is better than other CPSO algorithms in the comparison of various CPSO algorithms.In the high dimensional and complex problems,the CPSO algorithm rapid decline to the smooth region and leads to convergence stagnation in the early.The aggregation of a large number of particles leads to the local optimum of the algorithm.This paper presents an improved CPSO algorithm,combined with the gradient descending method.The local search ability of gradient descent method is used to make the CPSO algorithm out of stagnation and continue to explore.The results show that the proposed method improves performance in most cases,and the optimal result is obtained by using the hybrid method of hierarchical CPSO algorithm and gradient descent method.