The Reseach And Application Of Stochastic Gradient Descent And Dual Coordinate Descent Algorithm

With the advent of big data, the field of pattern recognition has emerged in a mass of high-dimensional large sample data. It is challenged for the processing speed and recognition accuracy of classification algorithms. This paper mainly studied the convex optimization algorithm for high-dimensional large sample data processing relatively quickly, and carried out the application experiment. It has obtained the good effect.Fristly, This paper analyzes the principle of gradient descent method and its advantages and disadvantages. On this basis, introduced the stochastic gradient descent method, and analyzed its convergence and solving steps in detail. Then, combined with multi class SVM, we have carried on the simulation experiment and daily activity recognition experiment and obtained the good effect.Second, in view of the shortcoming of gradient descent method iterative calculation quantity is big, not suitable for high-dimensional data and the continuous gradient of stochastic gradient descent method is change bigger, may point to the opposite direction, we lead to the half stochastic gradient descent method. This paper has deduced the algorithm steps of half astochastic gradient descent method in detail in this paper, and analyses its convergence and the optimal parameter selection. Combined with Logistic regression model, we conduct the simulation experiment and applied to the problem of schizophrenia diagnosis classification. The experimental results show that the method has achieved good effect to the diagnosis of schizophrenia and the classification accuracy is 91.41%. It is higher than that of the stochastic gradient descent method and has a fast calculation speed.Finally, the article analyzes the basic principles of coordinate descent method, and extends out the dual coordinate descent method. Then analyzes the main computing steps and convergence of dual coordinate descent method, and combined with binary Logistic regression and the maximum entropy model, separately from their dual problem to solve. Whereafter, we conduct the experiments in the benchmark data sets and natural language processing data sets, and compare the convergence and accuracy with other algorithms. It has obtained the good effect.