The Research Of Online Modeling Based On Gaussian Process

As a non-parametric probability model, Gaussian Process has already become an important method of machine learning field. Compared with other modeling methods, GP has the advantage of giving both the prediction and the confidence degree of the prediction, which is also called variance. Based on statistical learning theory, GP can be used for classification and regression, and has unique advantages in resolving small sample, nonlinear problems.Compared with other modeling methods, GP has the advantage that has fewer parameters to optimize, but its optimization process of parameters is still the most time-consuming part of the system identification work. The most common training method of optimizing hyper-parameter is the conjugate gradient method, which needs to compute the Hessian matrix. In iterative processes of online algorithm, the Hessen matrix needs a lot of time to compute, so the conjugate gradient method is not appropriate for online algorithm in the requirement of real time. Meanwhile, the computational complexity of Gaussian process is closely associated with the size of sample set. If the number of training sample is larger, the covariance matrix of GP model and its inverse calculation would need much computing resources and time. So, for the online training algorithm, it will not suitable for the requirement that directly calculating the precise value of the matrix's inverse.Based on Riemann space, Adaptive Natural Gradient method (ANG) has the advantage of simple in calculation and closed to efficient Fisher method, when compared to the standard Gradient method. So for solving the problem that is mentioned above, this paper puts forward the online GP modeling algorithm based on Adaptive Natural Gradient, which will use the Adaptive Natural Gradient method in optimization of the online GP model. Unlike the bulk learning method, Incremental learning method is different and can add sample on the iterative process of learning, then use operation result of last iteration process in the present process to reduce the computational complexity. So, the algorithm can rapidly shorter the training time when the new sample added to the modeling training process. Online Gaussian process algorithm constantly put new training data to training set, by adjusting model parameters, implement Gaussian process model of real-time optimization. So, we can not only improve the training speed, but also the adaptiveness.The algorithm is utilized in Mackey-Glass system and Continuous Stirred Tank Reactor (for short, CSTR) system to modeling. The simulation results show that the nonlinear system model satisfied the requirements of real time and precision.Finally, am at resolving data redundancy problem which appeared in the algorithm of online Gaussian process modeling, this paper use the correlation analysis, data standardization and adjust the probability density threshold method of data pretreatment. So we can get more suitable data for modeling, so as to reduce redundancy and further to improve the training speed.