Research On Deep Bayesian Latent Variable Models And Their Inference Methods

Deep learning plays a vital role in the evolving third wave of artificial intelligence because of its powerful data fitting,representation learning ability,and wide range of applications.However,several properties of the existing deep learning models hinder their development to some extent,such as massive data-driven,insufficient robustness and interpretability,and inability to represent uncertainty.In contrast,probabilistic graphical models have good performance in dealing with interpretability,uncertainty,and robustness by introducing the probabilistic framework and latent variable distribution.Therefore,deep Bayesian learning,which draws on the strengths of probabilistic graphical models and deep learning,has become an immensely valuable research direction in recent years.Generally,deep Bayesian learning can be divided into Bayesian neural networks and deep Bayesian latent variable models from the perspective of the fusion mode of probabilistic graphical models and deep learning.This dissertation mainly focuses on the study of deep Bayesian latent variable models and their inference methods based on two core issues uncertainty representation and adversarial robustness.Specifically,this dissertation focuses on designing the fusion style and efficient inference methods that seek to integrate the advantages of latent variable models and deep learning and conducts a progressive discussion in six chapters through three typical tasks: link prediction for graph data,image classification,and image generation.The main contributions can be summarized as follows:First,for the problem of the lack of uncertainty representation of the neural tensor network,a Bayesian neural tensor network is proposed in the link prediction task for graphs.Specifically,this method focuses on link prediction in knowledge graphs,which are a typical type of graph data structure.Firstly,this method denotes entities and relations as latent variables that obey Gaussian distributions rather than deterministic vector representations,capturing the uncertainty in the representation of entities and relations,and also introducing prior knowledge for entities and relations.Secondly,the multivariate Bernoulli likelihood function is utilized to represent the fact that there is a relationship between entities,which depicts the uncertainty of predictions.Then,a multi-layer perceptron is used to represent the relationship between entity latent variables and relation latent variables,depicting more complex interactions.Finally,this method employs a stochastic gradient variational Bayesian framework based on the reparameterization trick for achieving efficient approximate variational inference.Experimental results show that this method can capture the uncertainty representation of entities and relations and provide confidence in predictions along with good performance.Second,for the problem of low adversarial robustness of the deep Gaussian process model,an edge-enhanced deep Gaussian process model is proposed in the image classification task.Specifically,this method adds an edge enhancement layer in front of the deep Gaussian process model.Facing the scenario that the edge detection module is not differentiable,a straight-through estimator is employed to achieve end-to-end training.Then,this method employs a stochastic variational inference method based on inducing points to enable sparse approximate inference of Gaussian processes.Finally,by incorporating the adversarial training strategy,this model leads to more significant performance boosts.Experimental results show that this model can explicitly compensate for the shape information of images in an insensitive way,thus boosting the robustness of image classification.Third,for the problem of low adversarial robustness of variational autoencoders,a regularization method based on self-supervised learning is proposed.This phenomenon can largely be attributed to the non-smoothness of the encoder.To address the lack of smoothness of the encoder,a contrastive learning regularization term is introduced for the variational autoencoder,so that the latent representations of the original examples are close to those of the corresponding adversarial examples,but far from those of the other adversarial examples.Then,considering the distribution of latent variables,this method designs the exponent of SKL distance to measure the similarity between latent representations.Experimental results show that this method can explicitly constrain the learning process of latent representations,thus improving the adversarial robustness of image generation.