Research On Least Squares Twin Support Vector Regression

Twin support vector regression(TSVR)is an effective data prediction method in the field of machine learning.During the training process,TSVR needs to solve the problem of quadratic programming.Therefore,the efficiency becomes lower when training large-scale data.The least squares twin support vector regression(LSTSVR)converts the training process into solving linear matrices,which greatly reduces the computational complexity.However,the LSTSVR has its problems.Firstly,LSTSVR ignores the structural information of the samples and its prediction performance can be further improved.Furthermore,the least squares method in LSTSVR is sensitive to outliers.In addition,the parameters of LSTSVR are commonly selected by empirical method or exhaustive search method,and their optimization efficiencies are rather low.In this thesis,we attempt to improve the prediction performance,lower the influence of potential outliers existed in samples and optimize the parameters of LSTSVR.The main contributions are summarized as follows:In order to solve the problem that LSTSVR does not consider the structural information of samples,we proposed a least square twin projection support vector regression(LSTPSVR).First,the upper and lower bound functions are determined by finding a suitable projection axis,which are determined by minimizing the projection variance of the sample set.Secondly,the variance of the sample set is computed by the empirical variance of the input sample and its empirical correlation coefficient,which means that the structural information of data is introduced.The experimental results show that LSTPSVR can effectively improve the prediction performance.In order to address the problem that the potential outliers in the sample may have a great influence on the prediction performance of LSTSVR,and considering that the margin distribution plays an important role in promoting the generalization performance of the training model,we proposed an isolation forest-based least squares twin margin distribution support vector regression(IFLSTMDSVR).Firstly,the anomaly score of each sample is determined by the anomaly score mechanism of isolated forest.The samples of higher anomaly scores will be assigned with smaller influence factors so as to weaken the influence of outliers on regression.Secondly,to further improve the generalization performance,the average margin and variance margin are integrated into the objective function by embedding margin distribution information.The experimental results demonstrate that IFLSTMDSVR can effectively reduce the influence of outliers on the prediction performance and improve the generalization performance.Parameter selection is closely related to the performance of the algorithm.Grey Wolf optimization(GWO)algorithm has the advantages of fast convergence and simple structure.In order to address the parameter selection problem of IFLSTMDSVR,we attempt to utilize GWO to optimize the parameters of IFLSTMDSVR.The GWO algorithm considers the root mean square error or mean absolute error as the fitness function.Based on the position update mechanism,the optimal parameters can be obtained within finite iterations.Then,the IFLSTMDSVR optimized by GWO algorithm is applied to the soft sensor measure of penicillin fermentation process.The experimental results indicate that the proposed method can find the suitable parameters within a shorter time,and has higher optimization efficiency as well as better prediction performance.