Research On Algorithms For Large Scale Sparse Least Squares Support Vector Machine

Least squares support vector machine?LS-SVM?is an important machine learning model and has been widely applied to the real world problems.One of the main shortcomings is that its solution is not sparse,which leads that almost of training samples have been used to the final decision and makes the prediction speed become very slower.With the rapid development of the Internet,the data scale has become enormously large,which leads that it is extremely difficult to train the model on a single machine.The reason is that on the one hand,the single machine cannot store huge amount of data,on the other hand,limited by the computing ability of the single machine,the complicated tasks would take a lot of time.Therefore,it is an important issue how to obtain the spare solution of large scale LS-SVM.Based on the₁ regularized regression,a sparse algorithm is proposed for LS-SVM in this study.Inspired by the ensemble learning method,Bootstrap sampling is first used to select training samples for large scale problems.Secondly,each of the₁ regularized regression model is parallel solved.Finally,the original solution of the large scale problem is obtained by the ensemble learning method.In order to verify the effectiveness of the proposed algorithms,some experiments are firstly conducted on the smaller regression and classification data sets respectively.The experimental results show that the proposed algorithm can obtain the sparse solution for the smaller scale problems without significantly degrading the test precision.The experimental results on large scale data sets show that compared with the existing large scale algorithms,spending less training time,the proposed algorithm can also obtain the sparse solution without significantly degrading the test precision.