Revised LARS And Its Application In Sparse SVM

Least square support vector machine(LSSVM)is a variant of support vector machine(SVM),which is widely used in classification and regression problems.In solving quadratic programming problem,LSSVM can be transformed into a solution of the linear equations compared to SVM.LSSVM improves the training speed,but loses its sparseness,resulting in slow test speed.LSSVM is not suitable for large-scale data sets,thus it's important to sparse the solution of it.LARS is an efficient and fast method of solving the lasso problem,meanwhile,the solution of the lasso problem will be sparse.So in the paper the linear equations of LSSVM will be considered as a lasso problem.Then the principle of the LARS is used to solve it and to obtain its sparse solution,so as to achieve the purpose of sparsing the solution of LSSVM.The main work of this paper includes two aspects as follows.On the one hand,the data(coefficient matrix)whose columns are full rank will be required when we use the LARS to solve the lasso problem.However,the coefficient matrix corresponding to the LSSVM in the original space(P-LSSVM)does not satisfy this requirement.Therefore,a revised LARS method is proposed,the variables that may be selected which linearly related to the selected variables are excluded.The experiments show that the RLARS can avoid the selection of the approximate correlation variables under the appropriate threshold parameters.In other words,RLARS can solve the lasso problem whose coefficient matrix is not a column full rank.Based on it,we proposed a new sparse LSSVM method named as RLARS-LSSVM which uses the RLARS to solve the linear equations of P-LSSVM.This method can control the sparsity level of the solution(the number of support vectors)of P-LSSVM by controlling the number of iterations.In P-LSSVM,due to the constraints of memory and time,the kernel matrix of large-scale data sets can no longer be calculated by the definition.So we use reduced-rank approximate method to approximate the kernel matrix.Finally,the effectiveness of the algorithm RLARS-LSSVM is illustrated by linear classification and nonlinear classification experiments.On the other hand,the LARS is revised based on the principle of error minimizaiton,and a new method named as rank one revision(ROR)is proposed to solve the lasso problem.In the process of the interation,according to the geometric significance of the least squares,ROR updates directly the fitting value at each step,without calculating the specific direction and step size which reduces the amount of the calculation.And because the ROR has the property of selecting vectors which are completely linearly independent and the error decreasing in the iterative process.So this paper presented another new method of sparse LSSVM ROR-LSSVM,which used ROR to solve the linear equations of D-LSSVM.The method can also control the number of support vectors by controlling the number of iterations.In the process of the iteration,the trainig error gradually decreases to ensure the increasing trend of the test accuracy.Finally,the effectiveness of the algorithm ROR-LSSVM is verified by linear classification and nonlinear classification experiments.