Convergence Analysis Based On Momentum BP Algorithm

Artificial neural network has shown great potential in many areas of modern scientific development.BP neural networks using the error back-propagation algorithm(BP algorithm)as a learning algorithm are widely used.Adding momentum to the BP algorithm is also a very common method to improve the BP algorithm,which can be called the momentum BP algorithm.There are many factors that affect the learning efficiency of the momentum BP algorithm,including the number of layers on the network,the activation function,and the error function.In practical applications,the algorithm often performs better by trying to change some factors constantly,but convergence analysis often fails to keep up.In this paper,we analyze the convergence of a BP neural network algorithm with momentum terms containing multiple hidden layers.When the learning rate is constant and the momentum coefficient is adaptively changed under certain conditions,we give both the weak and strong convergence results of the algorithm,and give corresponding theoretical proofs for both convergence results.After that,we further analyze the convergence of the momentum BP algorithm with penalty term.Similarly,we also give the weak convergence result and strong convergence results of the algorithm,and theoretically prove the convergence results.