Analysis On Self Adaptive Learning Rate Algorithms For BP Neural Networks

Artificial Neural Network (ANN) is a new interdisciplinary which rises in recent years. It is a distributed parallel information processing mathematical model that simulates the conducts and features of biological neural networks. Neural networks are widely used in many fields, including pattern recognition, data mining, finance forecast, and so on.BackPropagation neural network (BPNN) is a feedforword neural network with error backpropagation and wildly used in lots of fields. Although BPNN is one of the most popular neural networks nowadays, its convergence speed is very slow and the training time is very long. To improve the performance of BPNN, Researchers have proposed many solving solutions. One kind of them is self adaptive learning rate algorithms. The algorithm is a quite simple and efficient improvement of BPNN. The performance of BPNN with different algorithms varies to the same problem.The convergence speed of BPNN can be efficiently improved by self adaptive learning rate. However, for the same problem there are obviously differences between different self adaptive learning rate algorithms. The convergence speed of different adaptive learning rate algorithm is compared and analyzed through numerical simulations for several benchmark problems.