Application Of Improved Least Squares Support Vector Machine

Support vector machine (SVM) is a particular machine learning algorithm which is basedon structural risk minimization principle. SVM aims to achieve a good trade-off between thelearning ability and the complexity from finite samples. As a modified version of standardSVM, least squares support vector machine (LS-SVM) has shown good performance when itdeals with the small samples, nonlinearity and local minima problems, but LS-SVM lackssparseness and robustness. We have also noticed that the hyper-parameters of LS-SVM needto be optimized. In order to solve the above problems of LS-SVM, some improved methodsare proposed as follows:(1) For improving the fitting precision and generalization ability of LS-SVM, the CSAalgorithm is proposed to optimize the hyper-parameters of LS-SVM. The CSA algorithm isused to handle multiple independent parallel simulated annealing optimization process, whichimproves the optimization information exchange. The acceptance temperature is employed tocontrol the variance of the acceptance probabilities of hyper-parameters which reduces theinfluence of the CSA algorithm to initialization parameters. Finally, the LS-SVM regressionmodel which is optimized by CSA algorithm is established to make some experiments basedon the field data. The simulation results show that the improved LS-SVM regression modelachieves good prediction performance.(2) In order to reduce the influence of the noise data on the robustness of LS-SVM model,the data is modeled and forecasted by iteratively reweighted least squares support vectormachine (IRLS-SVM). Firstly, weighted function iterations are increased to assure modelprocess robustness; Secondly, the IRLS-SVM hyper-parameters are optimized based onmethod which is combined by globally optimal CSA and local optimum method SM; Thirdly,the robust cross validation is used as CSA-SM algorithm objective function to improverobustness of IRLS-SVM model for hyper-parameters optimization process; Finally,numerical experiment is carried out by using gearbox data, and the result shows that theproposed method is effective.(3) In order to solve the sparseness problem of LS-SVM, two sparse models areestablished as follows:①Based on feature vector selection (FVS) method, a new model of sparse least squaressupport vector machine (SLS-SVM) is proposed. Firstly, a subset of feature vectors is definedin feature space to reconstruct all the training samples linearly. Then, the sparse featurevectors are used as support vectors, and the SLS-SVM model is obtained. Finally, numericalsimulation study and simulation experiment of pantograph-catenary system are carried out. The simulation results show that SLS-SVM not only has good forecast precision, but alsoachieves highly sparse support vectors, for which the prediction speed is accelerated.②The iteration sparse least squares support vector machine (ISLS-SVM) is establishedby importing the L0-norm regularization term into the LS-SVM’s objective function, and thesupport vectors can be sparse asymptotically. However, there are extra linear systems need tobe computed. Then, the fast leave-one-out cross-validation is introduced to reduce thecomputation cost of initial setting for this sparse model. Furthermore, with a series offormulations, the linear systems which are quite time-consuming to solve are solvedeffectively by using Cholesky factorization. Finally, the simulation results show the efficiencyof the improved ISLS-SVM regression model.