A Study On Iteration Algorithm With Momentum Term

With the quickening of the science and technology development speed,the neural network calculation and large sparse linear systems are widely used in the field of scientific computing and numerical algebra iterative algorithm.And the research of neural network computing and numerical algebraic iterative algorithm has become a very important topic.In order to improve the efficiency of scientific calculation,it has become a very important research topic how to optimize the existing iterative algorithm.It is now used in physics to accelerate neural network and numerical algebraic iterative algorithm,so as to improve the convergence efficiency and reduce the consumption of computer memory.In this paper,several kinds of iterative algorithms are studied to study the momentum term to optimize the algorithm,and the thesis is divided into three parts.First,this chapter probes into a gradient algorithm of the fixed learning rate for Sigma-Pi-Sigma neural network.The momentum renewal is introduced in order to improve its convergence.Under the situation of variable momentum factor,the gradient algorithm with momentum is studied to confirm its convergence and the result of error function monotony is offered as well.The weak convergence theorem for this algorithm is obtained.Secondly,a new generalized SOR method is applied to improve the convergence speed and computer memory consumption of the algorithm,and verify the correctness and validity of the algorithm by numerical experiments.Thirdly,on the basis of a new iteration algorithm,the momentum term is added to solve the generalized inverse of the matrix,and the convergence is studied deeply,and the optimal parameters of the algorithm are given.Numerical experiments are used to verify the effectiveness of the proposed algorithm.