Research On Accurate Incremental Online V-Support Vector Regression

The Support Vector Machines (SVM) is a machine learning approach which can solve the learning problem of small scale samples effectively. The theoretical foundation of SVM is statistical learning theory. In recent years, SVM has been widely used in many fields including data mining, biomedical, image processing, pattern recognition, artificial intelligence. The learning of training samples is one of the key problems of SVM. In practical regression problems, such as bioprocess, network data monitoring, financial data analysis, time-series prediction and so on, the samples are usually provided incrementally in online scenarios, in the extreme case, one sample at a time. In these cases, the traditional batch learning algorithm will lead to lower model prediction accuracy and worse robustness due to lack of real-time, whereas the incremental online learning algorithm will offer a feasible path for solving the problems mentioned above.v-Support Vector Regression (SVR) is an effective learning approach for regression. In comparision to ε-SVR, it has the advantage of using a parameter von controlling the number of support vectors and adjusting the parameter ε of insensitive loss function automatically. However, the dual problem of v-SVR is more complex than the dual problem of ε-SVR. To date, there is no research on the incremental online learning algorithm which is specially designed for v-SVR. Based on the accurate incremental online C-support vector classification learning algorithm which was proposed by Cauwenberghs and Poggio (referred to as C&P algorithm), the accurate incremental online v-SVR learning algorithm as well as its feasibility and finite convergence, and the application of the proposed algorithm in the soft senor of fermentation process are mainly researched in this dissertation. The main works of this dissertation are listed as follows:(1) The equivalent formulation of v-SVR is proposed to address the two complications existed in the dual problem of v-SVR. The first one is that the box constraints are related to penalty parameter and the length of the training sample set; the second one is that the dual problem of v-SVR has an extra inequality constraint in comparison to the dual problem of ε-SVR. The two complications were addressed by multiplying the length of the training sample set and substituting the inequality constraints with inequality constraints. The equivalent formulation of v-SVR is the cornerstone of the following research works.(2) To solve the infeasible updating solution path problem of the v solution path for v-SVR, an effective v solution path for v-SVR is proposed. Based on the equivalent formulation of v-SVR and its Karush-Kuhn-Tucker (KKT) conditions, the strategy of introducing a new variable △(?) and an extra term κ△p could solve the contradictions and exceptions effectively during the adiabatic incremental adjustments. Finally, the proposed algorithm could fit the entire v solution path within finite number of iterations. Theoretical analysis and simulation results show that the proposed algorithm is feasible and effective.(3) To solve the problem of unable to generate an effective initial solution which is caused by the extra linear term introduced in the objective function of the dual problem of the v-SVR and the infeasible updating solution path problem when directly applying the C&P algorithm to online learning of v-SVR, an accurate incremental online v-SVR learning algorithm is designed based on the equivalent formulation of v-SVR and its KKT conditions. The proposed algorithm was composed by three steps:the first one was prior adjustments; the second one was relaxed adiabatic incremental adjustments; the third one was accurate restoration adjustments. Theoretical analysis proved the feasibility and finite convergence of the proposed algorithm. The simulation results on benchmark datasets further verify the conclusions of theoretical analysis. Furthermore, the proposed learning algorithm is of higher computation efficiency than batch learning algorithm.(4) Due to the fact that some important biological state variables are rather difficult to be online measured during fermentation, the accurate incremental online v-SVR learning algorithm was applied in the soft senor of biomass concentration and product concentration of glutamic acid fed-batch fermentation process, which realized the online estimation of biomass concentration and product concentration. The simulation results demonstrate that the accurate incremental online v-SVR learning algorithm is suitable for the online estimation of key state of fermentation process.