| Support vector machine （SVM） is a novel machine learning method based on statistical learning theory and kernel function, and be widely used in application. Least squares support vector machine （LSSVM） is a modified algorithm of the standard SVM, has inherit good characteristics of SVM,such as small sample, simple construction, global optima and good generalization property. However, many aspects of LSSVM are still incomplete, as LSSVM suffers from lack of sparseness and robustness. In this paper, we are mainly devoted efforts to improve sparseness and enhance generalization performance.The main works of this paper is described as follow:1. In order to realize LSSVM’s sparseness, we proposed sparse l0-norm least squares support vector regression in primal space algorithm. Firstly, we proposed a sparse learning framework to reduce the number of support vectors by adds the l0-norm regularization term into the optimization problem of LSSVM, then solved in the in primal space, we get the sparseness in a iteratively way. Experimental evaluation on one group of artificial dataset, UCI databases and Gas furnace example demonstrated that the proposed algorithm yield significantly fewer support vector and good generalization performance.2. In order to improve generalization performance of LSSVM, we proposed a novel selective ensemble least squares support vector regression. At first, the gauss noise is injected to the original training dataset to generate the new training dataset. Then, the LSSVM algorithm is used to construct the sub-models in the ensemble model. At last, the ensemble model is obtained by compressive sensing using orthogonal matching pursuit algorithm. The proposed algorithm is used to establish the soft sensor model of the dry point of aviation kerosene. The simulation results show that the proposed algorithm has better performance both in approximation and prediction.3. Again, aiming to improve generalization performance of LSSVM, we proposed multiple kernel learning least squares support vector regression based on lP>1-Norm constraint. We firstly introduce the framework of multiple kernel learning, and then deduce the lp（p>1）-Norm constraint multiple kernel learning, based on the target function of LSSVM, we give out two Wrapper methods to solve the proposed algorithm, both the training method are two-step methods, use an iterative approach where each iteration, first we update the combination function parameters while fixing the LSSVM parameters,and the we update the parameters of LSSVM while fixing the combination function parameters, these two steps are repeated until convergence. Simulation on one-variable function and multivariable function has implied that the proposed algorithm is useful and outperforms the traditional LSSVM for generalization performance. |
