Fast Solving Algorithms For Two Classes Learning Machines With Nonparellel Hyperplanes

As a statistical machine learning method with nonparallel hyperplanes,twin support vector machine and its extensions have achieved fruitful research results in the binary classification problem.However,it is faced with some problems such as model selection and fast solving when it is extended to multi-class classification and regression problems.In view of the above some problems,two classes of nonparallel hyperplanes learning ma-chines are studied and their fast solving algorithms are developed.The main innovations of this paper are as follows:Given that model selection problem of multi-class classification twin support vec-tor machines uses "1-versus-1-versus-rest" strategy,based on model transformation and data partition,it is proved that the solutions of the two sub-optimization problems are piecewise linear on the regularization parameters,and the entire regularized solution path algorithm is developed accordingly.System of linear equations and block matrix theory are applied to prove piecewise linear solutions.Six types of events are defined to seek the first event and then the solution path algorithm is designded.In addition,the initializa-tion conditions of the solution path are given.It is proved that when the regularization parameter ?((?))tends to infinity,the lagrange multipliers ?_i(?_i*)and ?_k(?_k*)are 1.Simula-tion results based on UCI dataset show that the proposed method can achieve satisfactory classification performance and reduce the training time.For the regression problem with noise,a new weighted mechanism and weighted least square twin support vector regression are proposed by using the information of sample responses,which reduces the influence of noise on prediction accuracy.Futhermore,the offline and online learning algorithms about weighted least square twin support vector re-gression are proposed.The simulation results based on artificial datasets and benchmark datasets with noise show that the proposed method is better than the least square twin support vector regression in terms of the sum squared error,the ratio of sum of squared error to sum of squared deviation,and the ratio of interpretable sum of squared deviation to real sum of squared deviation of testing samples.