| Pair-shifted projection twin support vector regression（PPTSVR）is an efficient data prediction algorithm for solving regression problems.PPTSVR only needs to solve two small-scale quadratic programming problems during the training process,and the training speed is around four times that of support vector regression（SVR）.Therefore,in recent years,PPTSVR has gradually become a hot topic in the field of machine learning.However,PPTSVR ignores the influence of samples located at different positions on hyperplane construction,which can weaken the fitting performance of the algorithm if there are potential outliers in the samples.Moreover,PPTSVR utilizes L2-norm to measure the distance from the projection of sample points to the projection center,which can amplify the influence of potential noise or outliers.This thesis focus on improving the fitting performance and training speed of PPTSVR.The main contributions are summarized as follows:In order to solve the problem that PPTSVR ignores the influence of samples located at different positions on the construction of hyperplane during the training process,a weighted smooth pair-shifted projection twin support vector regression（WSPPTSVR）is proposed.Firstly,isolation forest approach is utilized to assign different weights for each sample,and the influence of outliers on the hyperplane construction is weakened by assigning them with tiny weights.Secondly,in order to find the optimal hyperplane directly in the original space,the plus function is adopted to convert the constrained optimization problems into unconstrained ones and the Sigmoid smooth function is utilized to smooth the objective function.It is proved that the objective function is differentiable and strictly convex at any order.Then,the Newton iteration method is employed to solve the unconstrained optimization problems in the primal space.The results show that WSPPTSVR can effectively eliminate outliers in samples and improve the generalization performance.In order to solve the problem that the use of L2-norm to measure distance in PPTSVR can amplify the influence of potential noise or outliers in the samples on the hyperplane construction,a L1-norm loss based pair-shifted projection twin support vector regression（L1-PPTSVR）is proposed.The proposed algorithm utilizes L1-norm instead of L2-norm to measure the distance from the projection of sample points to the projection center,so as to reduce the influence of potential noise or outliers on the construction of hyperplane.L1-PPTSVR determines the optimal hyperplanes by minimizing the L1-norm-based losses.Then,by introducing appropriate slack vectors,the kernel matrix inverse is avoided and the storage cost is saved.The results show that the proposed L1-PPTSVR has better prediction performance and anti-interference ability in comparison to the existing representative regression algorithms.In order to solve the problem that L1-PPTSVR has high time complexity when training large-scale data sets,an improved L1-norm loss based pair-shifted projection twin support vector regression（IL1-PPTSVR）is proposed.Firstly,in order to reduce the time complexity,the density-dependent quantization scheme is employed to sparse the training sample sets.Secondly,due to the introduction of density-dependent quantization scheme,there are at least five parameters in IL1-PPTSVR.Therefore,to further accelerate the training speed of the proposed algorithm,the chaotic cuckoo search algorithm is adopted to optimize the parameters.The results show that IL1-PPTSVR has faster training speed on large-scale data sets than L1-PPTSVR. |
