An Adaptive Levenberg-Marquardt Method And Its Convergence Analysis

The Newton method and the Gauss-Newton method are classical methods for solving systems of nonlinear equations,however when the Jacobian matrix is singular or near singular,these two methods may fail.To address this drawback,Levenberg and Marquardt independently proposed the Levenberg-Marquardt method,which effectively avoided the drawback of the Jacobian matrix singularity or near singularity by adding a parameter ?k to make JkTJk+?kI transform a positive definite matrix.In order to make use of the known function values,Yamashita and Fukushima proposed a Levenberg-Marquardt method that set the function value as the LM parameter.Inspired by this idea,this paper presents an adaptive Levenberg-Marquardt method,which improves the efficiency of the algorithm,because it considers the LM parameter ?k?1,the sequence {xk} far away from the solution set X*is avoided.At the same time,because the non-singularity condition of the Jacobian matrix is strong,it is generally difficult to satisfy,so when this paper discusses the convergence,it uses a relatively weak local error bound condition.Then in order to expand the application of the LM method,the convergence condition is extended to the Holderian error bound which is more general than the local error bound.The main contents are summarized as follows:First,the local error bound is defined and the local convergence is proved under the local error bound.It converges quadratically,when ?Fk??1;it converges linearly,when ?Fk?>1.Also,the algorithm of the adaptive Levenberg-Marquardt method with Armijo line search is given.Its global convergence is proved by combining the local error bound.Second,the Holderian error bound is defined,where the local error bound is a special case of the Holderian error bound.Since the value of the order ? is different,its local convergence is also different.In the case of ?Fk? ?1,it converges,when? ?(0,1/2);it converges linearly,when ?=1/2;it converges superlinearly,when ? ?(1/2,1);it converges quadratically,when ?=1.In the case of ?Fk?>1,it converges,when ? ?(0,1);it converges linearly,when ?=1.Further,its global convergence is proved.Finally,numerical experiments are carried out by using examples of nonlinear least squares problem and systems of nonlinear equations to verify the effectiveness of the adaptive Levenberg-Marquardt method.Numerical experimental results show that the adaptive Levenberg-Marquardt method is effective.