On Issues And Applications For Least Squares Support Vector Machine

Support Vector Machine(SVM) is a newly developed mathematical model based on the Statistical Learning Theory, which has tremendous potentialities for application in the field of automatic control. However, SVM, as a relatively young technique, is still imperfect. So it is vital to further develop and consummate its theory, method and extend its application. For this purpose, this dissertation focuses on a modified model of SVM called least squares SVM. The main research results are concluded as follows:(1) A guideline for the choice of regularized parameters of LS-SVM is provided by theoretical analysis. This result also explains why the ideal result can be obtained even if the kernel matrix corresponding to some learning problems is non-positive definite. An advanced learning algorithm is necessary for the LS-SVM because its learning efficiency is too low to meet the actual needs when the scale of leaning problem is slightly larger. For this purpose, we first show the optimality conditions of LS-SVM, and establish its equivalence to a reduced linear system. Further, a novel LS-SVM learning algorithm based on improved PCG is proposed. At the same time, we also proved theoretically that the new algorithm is of faster convergence rate. Finally, experimental results show that computing speed of the new algorithm is faster than that of the original PCG algorithm, under the condition of the same correct rate.(2) To improve the qualities of dynamic system identification, a novel multi-dimensional support vector wavelet kernel function was proposed, i.e. modified L-P wavelet kernel function. It is proved that this kernel function satisfies translation-invariant kernel condition and can be used as a kernel function for SVM (Support Vector Machine). This function is not only an orthogonal function, but also is espe-cially suitable for local signal analysis, signal-noise separation and detection of jumping signals, thus enhances the generalization ability of the SVM.(3) Using modified L-P wavelet function as the support vector kernel function, the Least Square Support Vector Machine with modified L-P Wavelet Kernel was proposed. Simulation results show that the Least Square Support Vector Machine with proposed modified L-P wavelet Kernel function is better than that with L-P wavelet kernel or Gauss kernel in modeling and approximation abilities, and more adaptive to engineering application.(4) Suykens et al. (2002) described a weighted least-squares formulation of the support vector machine for regression problems and presented a weighted algorithm for robust approximation under the model bestirred enough. In this study, we present a novel method for achieving robustness in least-squares support vector machine that is prone to over-fitting under the model bestirred deficiently, which takes into account the characters of Cauchy distribution function and the selection of different values for weighted factor based on statistical features of the prediction error. The superiority of this algorithm is demonstrated on a numerical regression experiment.(5) Combining the modified L-P wavelet kernel and weighted iterative least squares support vector regression machine model (WILS-SVR), a novel notion of weighted iter-ative least squares support vector regression machine model on wavelet kernel (WILS-WSVR) is proposed. The weighted iterative procedure can efficiently update a trained LS-SVM by means of chunking incremental updating and decremental pruning proce-dures where the data come in sequentially, and robustness is improved by the use of an additional weighted LS-SVM step. This method overcomes the drawback of sparseness lost within the LS-SVM and makes the LS-SVM amenable to adaptive online imple-mentation. Aiming at the characteristics of signals, a modified L-P wavelet kernel satisfying wavelet frames is introduced. The wavelet kernel can approximate arbitrary functions in quadratic continuous integral space, hence the generalization ability of LS-SVM is improved. The WILS-WSVR is applied to non-linear regression modeling and Santa Fe Laser time series prediction. Experimental results show some advantages of WILS-WSVR over WILS-SVR on the modeling and generalization performance.(6) Fault diagnosis based on multiclass support vector machine is studied as an extension to the existed fault diagnosis domain. An extension of LS-SVM to the mul-ticlass case is emphasized. While standard SVM solutions involve solving quadratic or linear programming problems, the LS-SVM's corresponds to solving a set of linear equation, due to equality instead of inequality constraints in the problem formulation. The multiclass case that we discuss here is related to classical neural net approaches for classification where multi class are encoded by considering multiple outputs for the network, which not only has the excellent performance of generalization and recogni-tion accuracy, but also can solve the multiclass issue effectively. Experimental results indicated the proposed method could get good fault recognition results.