The Theory And Application Of Nonlinear Conjugate Gradient Algorithm

With the development of science and technology,especially the rapid development of internet and information technology,the scale of optimization problems faced by human beings is increasing.In order to solve such problems,many algorithms are proposed.Among them,the conjugate gradient algorithm is widely used because of its advantages such as simple iterative form,less calculation and storage space.In this paper,the conjugate gradient algorithm for solving nonlinear unconstrained optimization problems is further studied.Firstly,based on the analysis of the advantages and disadvantages of the conjugate gradient algorithm DY and HS,two conjugate gradient algorithms are designed based on their respective advantages.These two algorithms satisfy global convergence and have good numerical performance.The DY algorithm has a good theoretical convergence but the numerical performance is poor,while HS has a good numerical performance but the theoretical convergence is weak.In the first algorithm,we hybridize the HS and NLS-DY conjugate gradient algorithms to obtain the PHS conjugate gradient algorithm.The combination coefficient of its conjugate parameters is a non-fixed constant that can be automatically adjusted based on the gradient information of adjacent iteration points.The PHS algorithm can not only converge globally under the Wolfe-Powell line search,but also avoid continuous small steps,so that the algorithm has a good numerical performance.In the second algorithm,we combined the MHS and NLS-DY conjugate gradient algorithms to obtain the FHS conjugate gradient algorithm.The search direction generated by the Wolfe-Powell line search satisfies the condition of sufficient descent,which further satisfies the global convergence.Numerical experiments show that the FHS algorithm has efficient and stable numerical performance.Secondly,in order to get more practical algorithms,we study the conjugate gradient cluster algorithm and give three kinds of single parameter conjugate gradient cluster algorithms – WPRP,WHS and WLS.On the one hand,full descent and global convergence are studied for these three types of algorithms.On the other hand,three concrete conjugate gradient algorithms are obtained by selecting parameters through numerical comparison results – WPRP*,WHS*,WLS* algorithms.The average numerical performance of these three algorithms is better than that of other related conjugate gradient algorithms.Finally,we discuss the application of the improved conjugate gradient algorithm PHS and FHS in the time series ARIMA model.We use the PHS and FHS algorithms respectively to solve the ARIMA model parameter optimization estimation problem and obtain the PHS_ARIMA and FHS_ARIMA models.In addition,PHS_ARIMA and FHS_ARIMA models are used to fit and predict the corresponding real time series examples,which validate the rationality of the model and the effectiveness of the algorithm.