Nonparallel Plane Support Vector Regression Algorithm And Its Application

Support vector regression(SVR)is an effective and important tool to deal with regression problems in the field of machine learning and pattern recognition.Inspired by SVR,twin support vector regression(TSVR)was proposed in 2010.With the continuous research in depth,TSVR has gradually become one of the research hotspots of pattern recognition.However,in the real world,the sample data usually have noise or outliers due to the influence of various factors,which affects the prediction ability of TSVR.The influence is generally sensitive to noise or outliers and the robustness of the model is poor.This paper attempts to study TSVR from the aspects of improving training speed and the prediction accuracy and reducing the sensitivity of the model to noise or outliers,constructing a new loss function,and using the optimization theory and method to propose a robust TSVR,so that it can deal with practical problems more effectively.The specific research contents of this paper are as follows:1.Aiming at the problem of TSVR prediction accuracy and training speed,a novel nonparallel hyperplanes support vector regression(NNHSVR)is proposed by adding adjustment parameters to the objective function to constrain the upper and lower boundaries.The algorithm constructs two smaller quadratic programming problems,which is solved by over relaxation iterative method in dual space.The experimental results show that NNHSVR not only has better generalization performance,but also has faster training speed than TSVR.2.Aiming at the problem that TSVR is sensitive to noise or outliers,a novel robust twin support vector regression(Hε-TSVR)is proposed based on ε-insensitive loss function and Huber loss function construct a hybrid Hε loss function and the principle of structural risk minimization are considered.The algorithm is solved by Newton iterative method in the primal space,the experimental results verify effectiveness and robustness of Hε-TSVR.3.In order to further solve the problem that TSVR is sensitive to noise or outliers,inspired by Hε-TSVR and truncation loss function,a robust twin support vector regression(THε-TSVR)is proposed based on a smooth truncated Hε loss function which ε-insensitive loss function and Huber loss function construct and the principle of structural risk minimization are considered.The algorithm is transformed into a nonconvex optimization problem and solved by CCCP technology in the primal space.The experimental results verify effectiveness and robustness of THε-TSVR.