Multi-view Deep Gaussian Process Regression Model And Its Fast Training Method

Deep Gaussian process(DGP)is a popular probabilistic modeling method,which is powerful and suitable for function approximation and uncertainty estimation.It is widely used in various fields of machine learning.With the advent of the era of big data,the acquisition methods and types of data have continued to increase,and more and more multi-view data have emerged.However,the traditional DGP mainly deals with the modeling of single-view data and lacks the consideration of multi-view situa-tions.Therefore,this paper mainly studies the multi-view deep Gaussian process,and proposes a generalized multi-view deep Gaussian process regression model and a pre-trained multi-view deep Gaussian process model that makes the training process more efficient.Firstly,for multi-view data from different sources or composed of different types of features,this paper considers the data characteristics of different views and proposes an end-to-end multi-view deep Gaussian process(MvDGP)regression model to capture the characteristics of different views and model data in different views discriminately.In the process of inference,this paper makes two improvements on the basis of classi-cal probabilistic inference methods.One is that,unlike the general Bayesian training framework,it is not mandatory to assume that the latent variable layers are independent of each other,but the flexibility of the model assumption is retained to ensure the per-formance of the model.The other is the utilization of sparse variational inference and stochastic optimization to improve the capacity of modeling large-scale data.At the same time,the MvDGP model is not limited to the modeling of two views and can be generalized to scenes with more views.The depth of each view network can be flexi-bly set according to the characteristics of the data.Experimental results on real-world data sets show that the MvDGP model has significant modeling representation and un-certainty estimation capabilities for multi-view data and is feasible in large-scale data,and its performance outperforms commonly used multi-view learning methods and the state-of-the-art DGP models.Secondly,in order to further accelerate the training speed of the MvDGP model,this paper proposes a pre-trained multi-view deep Gaussian process(PreMvDGP)model.Taking advantage of the fact that the Gaussian process is equivalent to an infinitely wide neural network,and the fact that the training difficulty of neural networks and single-layer Gaussian process is lower than the deep Gaussian process,this paper designs a two-stage pre-training method for MvDGP model that is suitable for multi-view input.In the first stage of pre-training,a deep neural network with a similar structure to the MvDGP model is used to train the given data.In the second stage of pre-training,the values of each hidden layer trained in the first stage are used as the input and output of the corre-sponding single-layer Gaussian process model,and the parameters of each single-layer Gaussian are obtained by training.After sufficient training in these two stages,a suit-able set of initial parameters can be provided for the MvDGP model,thereby speeding up the training of the model and improving the generalization performance of the model.Experimental results show that the PreMvDGP model has outstanding performance as the MvDGP model in the classification task,which significantly outperforms common multi-view learning methods,the latest DGP models and deep neural networks,and is suitable for large-scale multi-view data.PreMvDGP model can make training reach convergence faster through the pre-training method.This model improves the com-puting efficiency while ensuring classification performance.Its computing speed has obvious advantages compared with the most advanced DGP models,and it is of great significance in practical applications.In summary,the proposed MvDGP model can effectively handle large-scale multi-view data,and the proposed PreMvDGP model further improves the computing perfor-mance.The work of this paper is the development and improvement of DGP model in multi-view scenarios and large-scale data.Experimental results verify the rationality and validity of the proposed model.