Research On Robust Support Vector Machines Based On Zeroth-Order Optimization

Support Vector Machines?SVM?as a kind of strong classification algorithm in the field of machine learning,provides sufficient theoretical support for scientific research and specific application.However,due to the structural characteristics of the hinge loss itself,when there is noise in the classification data,the classification effect of the traditional support vector machines can be significantly reduced,and its advantages are greatly reduced.Therefore,in order to solve the problem that the traditional support vector machines is sensitive to noise data,three different loss functions are proposed from the viewpoint of improving the loss function.From its own structure:these three loss functions limit the loss of the noise data to the[0,1]interval,thus reducing the influence of the noise data on the classification.In addition,the regularization term of SVM is combined with L₂ norm,margin mean term and L₁ norm to establish three different robust optimization models.Through the experimental verification,the three optimization models proposed in this paper have weakened the influence of the noise data,and robust of the SVM is improved.According to the three different optimization models,based on the idea of zeroth-order optimization,the corresponding solution methods are given in turn.Zeroth-order optimization is different from the traditional gradient calculation method,which does not need to calculate the accurate value of the gradient,but uses the function estimation value to replace the gradient,which avoids the complex gradient calculation.Regarding the three solving methods given:in the framework of zeroth-order optimization,with the technique of reduced variance,and introducing the techniques of weighted average,momentum acceleration,and proximal gradient in sequence,three efficient solutions are designed.For the improved robust support vector machines model,the corresponding algorithms are given respectively.The experimental results show that compared with the existing methods,the three methods proposed have some advantages in classification accuracy,reduced variance,convergence speed and so on.They can meet the needs of the proposed robust model and obtain better classification results on the given experimental data with different noise level.