Research On Gaussian Process Dynamical Systems

Sequential data are very common in the daily life. The data from adjacent time are associ-ated with each other, which is called dynamic. The data in different dimensions are dependent on each other, which is called dimensional dependency. Dynamic and dimensional dependency bring large challenges for modeling sequential data. A kind of effective methods is to capture dependencies both on the time and space under the framework of probabilistic model. In the frame of the probabilistic model, Gaussian process is an important and popular model, which provides an elegant nonlinear mapping modeling method. Lots of research on Gaussian pro-cess dynamical systems (GPDSs) are developed to solve the challenges for modeling sequential data. According to the model assumptions and learning methods, the existing GPDSs are di-vided into two broad categories which are Gaussian process dynamical model (GPDM) and variational Gaussian process dynamical system (VGPDS). In this dissertation, we are studying GPDSs deeply and putting forward novel and more reasonable GPDS models.First, we propose an improved GPDM including model learning algorithms and the condi-tional model to deal with missing data. The recently proposed GPDMs have been successfully applied to time series modeling. The learning algorithms for GPDMs are designed for model training with complete data. When data are incomplete, GPDMs reconstruct the missing data using a function of the latent variables before parameter updates, which, however, may cause cumulative errors. Therefore, we propose the improved GPDM to deal with incomplete data. In the improved GPDM, the improved algorithms can be used for learning the model with incomplete training data, and the improved conditional model can be used for recovering in-complete test data. We adopt the Bayesian framework and integrate out missing data to obtain the marginal distribution of observed data. Therefore, the improved GPDM can fully and prop-erly use the partially observed data. In order to verify the validity of the improved GPDM, we conduct experiments on incomplete motion capture data and make comparisons with the GPDM as well as k-NN, spline interpolation and VGPDS. The improved GPDM performs much better on both training with incomplete data and recovering incomplete test data.Then, we adapt the GPDM to a high-order GPDM to make traffic flow prediction. Traffic flow prediction which predicts the future flow using the historic flows is an important task in intelligent transportation systems (ITS). Efficient and accurate models for traffic flow predic-tion greatly contribute to the development of ITS. We adapt the GPDM to a fourth-order GPDM which is more suitable for modeling traffic flow data. Specifically, the latent variables in the fourth-order GPDM is a fourth-order Markov Gaussian process, and the weighted k-NN is in-corporated in the model to predict latent variables for efficient prediction. The future flow is estimated by the average of the results predicted by the fourth-order GPDM and k-NN. Com-pared with other popular methods, the proposed method performs best and yields significant improvements of prediction performance.Further, we propose a variational dependent multi-output Gaussian process dynamical system (VDM-GPDS). This work makes use of the previous achievements and works under the similar Bayesian framework. The VDM-GPDS is a kind of models for capturing complex dy-namics. It can better model multivariate sequential data, especially when the data from different dimensions are fully dependent. The outputs are dependent in this model, which is largely dif-ferent from previous GP dynamical systems. We adopt convolved multi-output GPs to model the outputs, which are provided with a flexible multi-output covariance function. We adapt the variational inference method with inducing points for learning the model. Conjugate gradient based optimization is used to solve parameters involved by maximizing the variational lower bound of the marginal likelihood. The proposed model has superiority on modeling dynamical systems under the more reasonable assumption and the fully Bayesian learning framework. We evaluate the model on both synthetic and real-world data. Various evaluation methods are taken on the experiments to demonstrate the effectiveness of our model, and encouraging results are observed.Finally, in order to make the VDM-GPDS more general, with capability of analyzing the sequential data and performing static data regression, we adjust the VDM-GPDS as a variational dependent multi-output multi-layer Gaussian process regression model (VDMM-GPRM). The VDMM-GPRM inherits the advantages of the VDM-GPDS. It can deal with multi-output or multi-task regression problems. The experimental verification with respect to the robot’s inverse dynamic problem proves that the VDMM-GPRM has superiority on regression tasks.Our work improves and perfects the existing GPDSs. In addition, we put forward a novel model for sequential data modeling, and unify the dynamical model and the static model under a framework. Different experimental tasks have verified the validity and superiority of the proposed methods.