The Doubly Regularized Support Vector Machine With A Globally Linearly Convergent Algorithm

In the classification problem, the L2-norm support vector machine whose effect is verygood for the two classification problem is a widely used tool. The L1-norm support vector ma-chine is a variant of the L2-norm support vector machine, due to the nature of the L1-norm,the L1-norm support vector machine has the property of automatically selecting variables, com-pared with the L2-norm support vector machine, it has many advantages, especially when thereare redundant noise variables and in high dimension problems.But these two support vector ma-chines have the limitations of themselves, when they are used to analyse the datasets with smallsample, high dimension and highly correlated variables, they sometimes can not get the idealeffect.The doubly regularized support vector machine, using the elastic-net penalty, a mixtureof the L2-norm penalty and the L1-norm penalty combined the advantages of these two supportvector machines. In this way, the doubly regularized support vector machine has the proper-ties of automatically selected variables, similar to the L1-norm support vector machine, andselection or remove the highly correlated variables together.However, because of the non-differentiable property of the L1-norm of the doubly regular-ized support vector machine and the inequality constraints, the calculation is complicated.Thestudying with the quadratic programming problem which has the equality constraints, prove thatas long as there is a non-zero solution, linearized Bregman algorithm for this problem enjoysglobal linear convergence and has an explicit rate of convergence.The doubly regularized sup-port vector machine problem can be transformed to the differentiable problem with the equalityconstraints, and then the linearized Bregman algorithm can be used to solve this doubly regular-ized support vector machine problem.The related numerical experiments on the simulation data and the real datasets show thatto solve the doubly regularized support vector machine problem with the linearized Bregmanalgorithm can reduce the computational complexity, improve the expected accuracy and get abetter classification accuracy.