Two Extensions Of Least Squares Twin Support Vector Machines And Their Online Learning Algorithms

Least squares twin support vector machines are obtained from traditional twin support vec-tor machines by modifying the two quadratic programming problems in least squares sense. Least squares twin support vector machines only need to solve two systems of linear equations, thus this learning machine obtains comparable classification accuracy and lesser computational time, which has attract much attention in machine learning and data mining area. To address the classification and regression problems with noise, we propose two extensions of least squares twin support vector machine and develop the online learning algorithms. The main work and innovations are as follows:(1) By introducing the double-weighted mechanism, we propose the double-weighted least squares twin bounded support vector machines to reduce the influence of noise on the classifi-cation accuracy and develop the online learning algorithms. Furthermore, a new pruning mech-anism which can avoid updating the kernel matrices in every iteration step for solving nonlinear model is also developed. Simulation results on three UCI data with noise demonstrate that the online learning algorithm for the linear double-weighted learning model can get the least com-putation time and considerable classification accuracy compared with least squares twin support vector machines, fuzzy weighted least squares twin support vector machines and offline learning algorithm for double-weighted least squares twin bounded support vector machines. Simula-tion results on three UCI data and two-moons data with noise demonstrate that the nonlinear double-weighted learning model can be effectively solved by the online learning algorithm with the pruning mechanism.(2) By introducing a new weight mechanism to regression problems, we propose the weighted least squares twin support vector regression to reduce the influence of noise on the prediction ac-curacy and develop the online learning algorithms. Simulation results on four UCI data with noise demonstrate that the online learning algorithm for the linear weighted learning model can get the least computation time and prediction error compared with least squares twin support vector regression and offline learning algorithm for weighted least squares twin support vector regression. Simulation results on the same data demonstrate that the online learning algorithm for the nonlinear weighted learning model can get good generalization performance.