Study Of The Convolutional Neural Network Optimization In LeNet-5 Based On Particle Swarm Algorithm

In recent years, the deep neural network model, both in the academic field and in the industrial sector, is a hot topic. In the academic field, at home and abroad of the top journals or conference have a lot of papers published about deep neural networks. In the industrial sector, foreign Google, Microsoft, Facebook, Amazon and other giants, as well as domestic Baidu, Alibaba, Tencent and other large companies have invested a lot of man power, material and financial resources to study the deep neural network model.Currently, the training of the deep neural network is usually using the the back propagation algorithm(BP), the core idea of the algorithm is gradient descent and it is easy to fall into local optimal solution, the more complex models, the bigger probability of falling into local optima. For the deep neural network model, parameter hundreds of millions, local optimum situation will become more serious. Particle Swarm Optimization(PSO) is a global search algorithm, which uses a plurality of particles according to a certain strategy parallel search solution space, while avoiding the trap of local optimal solution, which can get a better solution. Based on this theory, this paper attempts to use PSO algorithm to optimize the deep neural network, in order to alleviate the problem of local optima. Due to lack of laboratory resources, the paper chose the relatively simple structure of LeNet-5 deep convolutional neural network study. LeNet-5is advanced by deep learning master Yann LeCun, and has made much breakthrough on handwritten numeral recognition. It is the most classic deep convolution neural network model and the paper was also done experiments on handwritten digital standard library.Since the adjustment in the convolutional neural network PSO parameters is quite time-consuming, complex thesis on standard optimization functions do experiments to get better PSO parameters. Next, the paper uses a different version of the PSO algorithm to optimize LeNet-5 model. Through the experiment, we found that local versions of the PSO outperforms global version of PSO, which confirms the local versions of the PSO outperforms global version of PSO, which confirms the view that the local PSO precedence better ability to explore over the global PSO. In the local version of the PSO,RING topology is better than random topology. Finally, the paper used PSO algorithm to optimize Le Net-5 model on a different size data sets. Through experiments and found that Particle Swarm Optimization in small and medium-scale training sample obtained model, compared to BP algorithm has been greatly improved; but it did not get expected results in large-scale samples. Through the issue of deep learning network for PSO optimization method in large-scale sample, the paper targeted improved PSO algorithm to enhance its ability to explore. Meanwhile, the PSO algorithm and BP algorithm proposed a hybrid algorithm. The performance of the improved algorithm was validated in experiment at last.