Optimizing BP Neural Network Using Improved Simulated Annealing Genetic Algorithm

Abstract Genetic Algorithm is one kind of organic and adaptive random searching algorithm that simulates the evolution process and mechanism in nature to solve the optimization problems.Genetic algorithm was firstly given out by Professor J Holland from the United States in the early 1960s, the 1975 John Hollan formally established genetic algorithm.Genetic Algorithm develop from the Darwin and Mendel's theory of genetic variation. In the long process of biological evolution, by the genetic variation, in accordance with "survive the superior,eliminate the inferior and survival of the fittest". This rule evolved gradually from simple low-level development of complex biological senior. GA is a base in the natural genetic natural selection and the global optimization algorithm, using the mechanism of natural selection from the abstract by choice, crossover and mutation operator of the three genetic coding string parameters to operate. Due to a number of operations against a feasible solution for groups, so they can change their generations of parameter space parallel to the different areas domain search, and search more likely to find the optimal solution to the overall situation of direction. Due GA in the optimization process only use the evaluation function, not require the objective function differentiability, therefore, GA algorithm has overall, Parallel, fast, good adaptability and robustness of features. GA can solve many of practical problems. It has been in machine learning, pattern recognition, image processing, optimal control, combinatorial optimization and management decision-making areas to be a good application. Although genetic algorithms in many areas to be a very good application, but it ultimately is a new subject, the theory and method is not yet mature enough, the algorithm also own a number of shortcomings to be further improve and perfect. Although the genetic algorithm has been a lot of theoretical research, and to guarantee the genetic algorithm can converge to the whole the optimal solution to provide a theoretical basis. But genetic algorithms has easy precocious and slow convergence problems, and its local search capabilities is shortcomings and weak.Simulated annealing algorithm in the first put forward in 1953 by Metropolis, In 1983 Kirkpatrick, who was the successful introduction of combinatorial optimization field.The basic idea comes from solid annealing process, annealing is a physical process, when heated metal objects slow cooling, the temperature of the object will be to achieve the lowest temperature. Through observation and study, it was found that as the temperature dropped, the elements gradually stay in different temperatures, molecular achieve a stable state in the lowest temperature, molecular re-arrangement of a certain level of structure, and at a certain temperature, molecular achieve a stable state, a process that is repeated, and inner energy reached lowest.The simulated annealing algorithm made a simulation for the target function as the inner energy. Simulated annealing algorithm can find the optimal solution of the problem within a reasonable time .However, with the size and complexity of the problem the continuous improvement, Search process will be extended manifold.BP neural network is a multi-level feed-forward neural network and a most widely used artificial neural networks. As BP neural network learning algorithm is based on the gradient descent algorithm, inevitably have the following three drawbacks: (1) learning process slow convergence; (2) algorithm does not complete, easy to be trapped into local minima, and when high learning rate, which may have a concussion; (3) robustness of poor network performance for the network initially set up more sensitive.At present, some studies genetic algorithm, Simulated Annealing algorithm global search method has been applied to neural network optimization, and proved to be very good performance. Probability classical genetic algorithm search technology for the use of the optimal solution, has a strong global optimization capability. However, there is easy early, local optimization problems such as poor capacity. The simulated annealing algorithm in the search process, Some were randomized to receive inferior to resolve and has strong local search capabilities. Search process can not enter the regional expectations and the iterative slower.But the combined the two algorithms can improve genetic algorithm the local search capabilities.Through the above our genetic algorithm, BP simulated annealing algorithm and neural networks realized, we would like to find a better optimization algorithm to optimize BP neural network. In this paper, genetic algorithms and integration of simulated annealing algorithm both advantages Simulated Annealing applied parallel genetic algorithm optimization of the three-tier BP networks. (1) In the genetic algorithm can be seen as the introduction of a simulated annealing algorithm , on the genetic manipulation of individual implementation of the new strategy acceptable probability, enhance the global convergence of the algorithm, accelerate the evolution of the convergence rate of late. (2)It can also be considered in the simulated annealing algorithm introduced in the genetic algorithm groups operating thinking, algorithm in the solution space to start multiple local search, that is speeding up the algorithm search speed, can effectively improve the handling simulated annealing algorithm local convergence problems. (3)Improved simulated annealing genetic algorithm optimization BP neural network so that the network is more global and local search capabilities, and improve its search speed. In alphabetical identification as an example, Optimization of the method of verification, this paper presents the results show that the neural network algorithm parameters of the study results better than BP algorithm, classical genetic algorithm ; Optimizing the use of its network BP, its error rate decreased and convergence rate faster than the standard BP network, and on the learning rate adjustment requested less.