The Study Of Least Squares Support Vector Machine And Its Application In Industrial Process Modeling

The traditional approaches of industrial process modeling are mainly based on analytical models. This kind of method is suitable for the systems that we had known clearly how they work. For complex, nonlinear and dynamic industrial system, it is usually hard to analysis the mechanism of processes and to build up the mathematic models, moreover it will take us unbearable staff cost and financial cost. In most real production processes, an experienced human operator may have little knowledge about a complex system, but can still regulate control systems satisfactorily by observing the signals of inputs and outputs. Therefore, this behavior of mimicking the human ability by machine learning is an effective technique means.Studying the statistical classification or regression problem based on a given finite amount of samples, the researchers proposed the statistical learning theory (SLT). As a new technique for machine learning, the statistical learning theory is gaining more popularity due to distinguished properties and promising application performance. Support vector machine (SVM), one of novel machine learning methods, is to find a fine balance between the training error and the complexity of the learning machine. Because the formulation of SVM is guided by the statistical learning theory (i.e., structure risk minimization principle, Vapnik-Chervonenkis theory), these properties ensure that SVM can obtain global solutions instead of trapping in local optimal solutions under finite samples.In 1999, Suykens and Vandewalle proposed a modified version of SVM for classification, which is called least square support vector machine (LS-SVM) and resulted in a set of linear equations instead of a convex quadratic programming (QP) problem. Especially, LS-SVM has a significant advantage of the lower computational complexity than the other support vector machine formulations using linear or nonlinear mathematical programming. Therefore, Least squares support vector machine has shown an excellent classification or regression performance in many applications.In this thesis, the investigations are mainly focused on the application performances of LS-SVM, which involve the feature extraction, modeling, prediction and control problems. Several modification algorithms for LS-SVM are proposed here, that can be used to resolve the dynamic system issues respectively. The main contributions of the dissertation are as follows:1. In developing a SVM regression, the auto-relative or correlated features of input data could deteriorate the generalization performance of SVM. To resolve this problem, the first important step is feature extraction. A new feature extraction method, called dynamic independent component analysis (DICA), is proposed in this paper. This method is able to remove the major dynamics from the process, and to find statistically independent components from auto- and cross-correlatedvariables. To deal with the regression estimation, we combine the DICA with traditional support vector regression or least square support vector regression to construct multi-layer support vector regression. The first layer is feature extraction that has the advantages of robust performance and reduction of analysis complexity. The second layer is the SVM or LSSVM that makes the dynamic regression estimation. This modeling method is applied to estimation of process compositions in the simulation benchmark of the Tennessee Eastman (TE) plant. The simulation results clearly show that the estimator by feature extraction using DICA can perform better than that without feature extraction and with PCA, ICA, DPCA methods for feature extraction.2. To improve the modeling speed of LS-SVM, a recursive algorithm for training least squares support vector regression is presented in this paper. The recursive algorithm is based on the fixed, increased and decreased memory modes. Because three kinds of recursive modeling expression dispense with the need for calculating the matrix inversion, the online modeling speed of LS-SVM is accelerated in particular. Based on the criteria of minimizing the estimated error, a new adaptive modeling method is also proposed here. This modeling method adapted the LS-SVM to online learning problems and modeling of real industry modeling issue. Simulation analysis and the modeling of a typical plant for water treatment are also given .3. Because of equality instead of inequality constrains, the LS-SVM solutions lost the sparseness. To obtain a sparseness of support vector for least squares support vector regression, an algorithm called vector base learning (VBL) is proposed in this paper. And the concepts of base vector (BV), base vector set (BVS) and vector space are also introduced here. By calculating the angle between new sample vector and vector space, the criteria whether the measurement vector is one of the BVS is derived. As getting the new sample, the proposed algorithm on-line determines that the measurement is the support vector or not. This makes the solutions of LS-SVM have the feature of sparsity. Improving the modeling speed of LS-SVM, a recursive algorithm of increased memory mode for VBL algorithm is also proposed. Simulation analysis and the modeling of a typical plant for water treatment clearly illustrated the validity and feasibility of the presented method.4. Considering the impacts of 'excessive fitting' and 'unusual point' on identification, a weighted least squares support vector machine which based on the normal distribution function is proposed in this thesis. Its weighted rules mainly apply some statistical characteristics of the normal distribution faction. Confirming the parameters of the weighted rules according to the prediction errors, we can determine different weighted factor on each training variables. Because of emphasizing the real time properties of the sample, the weighted LS-SVM based on the normal distribution function has better robust and practicable ability than the known methods.5. Summarizing the results of the preceding chapters, a kind of nonlinear predictive control scheme based on the adaptive LS — SVM model is presented. A MPC-PID cascade control structure is built up. The inner control loop is PID controller, which has the performance of resisting the disturbance. As we know, PID has the characteristics of simple algorithm, fast sampling, and rapid noise immunity. The outer control loop is MPC controller, which lets the close control loop have the ability of higher robust. A simulation of CSTR process is given to prove the validity of the proposed method. The simulation results show that the nonlinear predictive control strategy based on LS — SVM model has a satisfactory performance.