Research On Conjugate Gradient Method In Statistical Learning

For the optimization algorithm in statistical learning,many scholars have always been attracted to study.This paper mainly focuses on statistical learning.When classifying a binary data set,two new optimization algorithms are proposed to solve the least square support vector machine and the regularized logistic regression model respectively.Under appropriate assumptions,the decline in the search direction of the algorithms and the convergence of the algorithms are discussed,and the effectiveness and feasibility of the algorithms are verified through a series of experiments.The main research work is as follows:First,transform the training of the least squares support vector machine into the solution of a linear system,and then transform the solution of the linear system into the solution of an unconstrained optimization problem.For this,a new conjugate gradient method is proposed based on the classical conjugate gradient method.Under certain assumptions,the proposed algorithm has the sufficient descent and global convergence,Finally,the numerical experiments show that the new algorithm is effective and feasible.Second,for the logistic regression model in the binary classification problem,the maximum likelihood estimation method is used to obtain the optimal parameters when solving the minimum value of the logistic loss function,and a₂l norm is added to the term of the loss function as a regularized logistic regression.The trust-region spectral conjugate gradient algorithm is used to solve the₂l norm logistic regression model.Under suitable assumptions,the sufficient descent and convergence proof of the new algorithm is theoretically given.In the numerical experiments,the numerical results of the new algorithm and the stochastic gradient algorithm are given when they are used to classify some data sets,and the comparison is made to verify the effectiveness and feasibility of the new algorithm.