Research On Twin Support Vector Regression

Twin Support Vector Regression(TSVR)is a novel machine learning method.TSVR only needs to solve a pair of small-scale quadratic programming problems and the constraints of each quadratic programming problem is only half that of Support Vector Regression(SVR).Therefore,TSVR runs four times faster than TSVR,and TSVR has become a new hot topic in machine learning field.In practice,the origin of the dataset is complex.The basic idea of TSVR is to minimize the fitting error.However,TSVR ignores the overall structure information,outliers and location information of samples.Hence,the generalization capability of TSVR may deteriorate in practical scenarios.In this thesis,we focus on improving the generalization capability,reducing sensitivity to outliers and parameter optimization of TSVR.The main contributions are summarized as follows:In order to solve the problem that TSVR is blind to the overall structure and location information of samples,we proposed a Structural Weighted Twin Support Vector Regression(SWTSVR).First,the proposed algorithm adds the covariance matrices determined by wards linkage clustering algorithm to the primal problem of TSVR.Then,the diagonal weighted matrices obtained by K-nearest neighbor algorithm are embedded into the primal problem of TSVR.Hence,the generalization capability of SWTSVR is improved because the sample information is reflected in the primal problem of SWTSVR.Secondly,to speed up the training process,the successive over-relaxation(SOR)algorithm is adopted to solve the quadratic programming problems.The experimental results show that the proposed SWTSVR algorithm has better fitting capability.In order to address the problem that TSVR is rather sensitive to outliers,we proposed a Fast Clustering-based Weighted Twin Support Vector Regression(FCWTSVR).Firstly,we utilize a fast clustering algorithm to quickly classify samples into different categories based on their similarities.Secondly,to reflect the prior structural information and distinguish contributions of samples located at different positions to regression,we introduce the covariance matrix and weighted diagonal matrix into the primal problems of FCWTSVR,respectively.In addition,to shorten the training time,we also adopt the SOR algorithm to solve the quadratic programming problems.The experiment results show that the proposed FCWTSVR has better prediction performance and anti-interference capability.The fitting capability of algorithm is closely related to the parameter setting.Fruit fly algorithm is a novel swarm intelligence algorithm,which has fast convergence speed and strong global search capabilities.To address the parameter selection problems,we attempt to apply the fruit fly algorithm to optimize the parameters of TSVR.The positions of the fruit flies are utilized to represent the parameters that need to be optimized.Then,the prediction performance is treated as the fitness function,and the fruit flies fly randomly to avoid trapping into local minimum.The highest fitness accuracy corresponds to the final positions of the fruit flies within finite iterations.The experiment results show that the fruit fly algorithm can find suitable parameters and the optimization time is shorter than other state-of-the-art parameter optimization algorithms.