Research On Multi-hyperplane Twin Support Vector Regression Algorithm And Its Optimization

Twin Support Vector Regression（TSVR or TWSVR）is a regression algorithm rooted in statistical learning theory,boasting a solid mathematical theory foundation.The algorithm solves two smaller-scale quadratic programming problems to derive two regression hyperplanes.It can efficiently handle high-dimensional data through kernel functions,obtaining a global optimum solution,effectively avoiding overfitting phenomena,and achieving significant success in small-sample scenarios.Relative to traditional support vector regression,twin support vector regression exhibits superior performance in terms of training time and fitting accuracy in small sample scenarios,making it a mainstream method for small-sample regression problems.However,there is still room for improvements and enhancements of twin support vector regression in small smaple scenarios,such as only considering empirical risk minimization,and most of them are applied to no-class scenarios,which is difficult to be applied to few-class and mutiple-class scenarios.In response to these issues and shortcomings,this paper first improves twin support vector regression,refining its theory and applying it to small-sample no-class regression scenarios.Secondly,this paper extends the inverse principle of twin support vector regression to multi-hyperplane twin support vector regression and applies it to small-sample multiple-class regression scenarios.Additionally,this paper generalizes the foundational principles of twin support vector regression to multi-hyperplane twin support vector regression and applies it to small-sample few-class regression scenarios.Finally,this paper extends the core idea of twin support vector regression to multi-hyperplane twin support vector regression,generalizing the theoretical advantages of twin support vector regression in small samples to large samples through the way of segmentation and integration of samples,and applies it to large-sample multi-class regression scenarios.The specific research content is detailed as follows:1.In response to the problem that twin support vector regression is susceptible to noise interference under the constraint of empirical risk minimization,the solution is not sparse,and may generate large errors,an asymmetric₁-norm loss-type wavelet transform weighted twin support vector regression is proposed.This algorithm uses asymmetric loss function and wavelet transform weighting to mitigate the impact of noise and outliers,uses₁-norm regularization to sparsify the objective function,and uses-type strategy to control the range of errors.Meantime,due to the poor global optimization ability of the salp swarm algorithm,cloud theory is used to enhance its population diversity,proposing cloud salp swarm algorithm for optimizing the parameters of the proposed algorithm.Experiments demonstrate that,compared to baseline algorithms,the proposed algorithm exhibits better performance in small-sample no-class regression scenarios.2.In response to the problem that twin support vector regression cannot utilize class labels of samples,leading to information waste when the samples exist in multiple classes,a multi-hyperplane twin support vector regression–multiple birth support vector regression is proposed.The algorithm generalizes the mathematical method of transforming support vector machines into support vector regression,and applies the generalized method to multiple birth support vector machine.It assigns class labels to each sample randomly,for each class of samples,it separately constructs a regression hyperplane that is far away from it,and simplifies the fitting process by averaging all hyperplanes.Additionally,the aquila optimization algorithm is employed to optimize the parameters of the proposed algorithm.Experiments demonstrate that,compared to baseline algorithms,the proposed algorithm exhibits superior performance in small-sample multiple-class regression scenarios.3.In response to the problem of multiple birth support vector regression in few-class scenarios,where the class labels random assigned to each sample significantly deviates from the true labels,and the averaging of all hyperplanes leads to a decrease in fitting accuracy,another multi-hyperplane twin support vector regression–fuzzy one-versus-all twin support vectore regression guided by fuzy clustering is proposed.The algorithm,at the clustering level,first obtains the class center by fuzzy density peak clustering,and then assigns the remaining samples by fuzzy K-nearest neighbor strategy;at the regression level,it first calculates the fuzzy weights for each sample through a fuzzy membership function,and then applies the fuzzy weights and the generalization method of transforming support vector machine into support vector regression to one-versus-all twin support vector machine.Based on the class labels provided by clustering,for each class of samples,the algorithm separately constructs a fuzzy regression hyperplane that is close to,and fits each sample using the hyperplane corresponding to its class of belonging.Additionally,the particle swarm optimization algorithm is employed to optimize the parameters of the proposed algorithm.Experiments demonstrate that,compared to baseline algorithms,the proposed algorithm exhibits superior performance in small-sample few-class regression scenarios.4.In response to the problem that the computation of the kernel matrix and its inverse matrix in the above algorithms takes a long time,the computational resources is consuming,and cannot extend its advantages to large-sample problems,the third multi-hyperplane twin support vector regression–least squares twin K-class support vector regression is proposed.The algorithm generalizes the mathematical method of transforming support vector machines into support vector regression into another form,and apply the generalized method along with the least squares strategy to twin K-class support vector machine.The algorithm assigns class labels to each sample randomly and constructs a pair of regression hyperplanes for each two classes.During construction,only the samples from these two classes are considered,while ignoring other samples.Therefore,when dealing with a large number of classes,the scale of the kernel matrix and its inverse matrix for each pair of hyperplanes in this algorithm is very small,allowing for direct computation,and the fitting process is simplified by averaging all hyperplanes,enabling the algorithm can segment the large sample into small samples and then integration them through the model’s inherent strategy.Additionally,the krill herd algorithm is employed to optimize the parameters of the proposed algorithm.Experiments demonstrate that,compared to baseline algorithms,the proposed algorithm exhibits superior performance in large-sample multiple-class regression scenarios.The dissertation has 50 figures,75 tables,and 152 references.