A New Improved BP Neural Network Based On Duplication Removal On Neighborhood

Neural network algorithm has been discussed by a lot of researchers both here and abroad, it plays a important role in artificial intelligence and heuristic algorithms, in application, it’s also used broadly in machine learning, pattern recognition and data mining.more than that because of it’s model ability, not only subject of compute science and technology, but also mathematic, physics, such subjects start use neural network for modeling theirs problems.we researched neural network algorithm with back propagation which is representative in neural network in this paper, we started from single neuron and linear problems to multilayer and non-linear problems, then derived the principle of BP neural network, during those procedure,few defects had been exposed to us, like the convergence speed of gradient descent is slow, training time is long.For those defects, we introduced much work of many researchers, one of the most important is that we regarded the minimum of cost function as a mathematic problem which is unconstrained and non-linear,then we imported the optimization theories, we used Newton’s way, quasi Newton’s way and conjugate descent method to decide the direction, then use line search or trust region to decide the step length, it solved defects of gradient descent method; for the selection of activation function,we introduced some specifications and some new equations which are better for convergence; learning rate and momentum factor also matter speed of convergence, some self-adapted way were illustrated; for the global convergence, heuristic algorithms support some methods to jump over minimum position, they are also described at this article.after that, we would clearly know that BP neural network algorithm can be separated into several parts, we can combine those parts to build a different and brand new one, and this would improve our understanding.Specially, gradient descent can’t get into the global minimum, we first looked for the reasons of this, then we imported the solutions that we can reach. After analyzing those, we used two ways which are hill climbing and genetic search to detect the solution space, and proposed our own way called duplication removal on Neighborhood to remove the duplicative solutions.we combined those algorithms with quasi Newton’s way and Levenberg-Marquardt method to improve BP neural network. After enough numeric experiments, we proved that this new BP neural network algorithm can convergence better.