Research And Application Of Probabilistic Generative Model With Variational Learning And Inference

Probabilistic generative model is an important model in the field of machine learning.It has been widely used in many problems and has achieved excellen-t results.This paper mainly focuses on some kinds of probabilistic generative models,studying their properties as well as their applications.Specifically,we mainly focuses on Deep Boltzmann Machine and Sequential Variational AutoEn-coder.The learning stage of both models requires the framework of variational learning.Variational learning provides a framework different from the Markov chain Monte Carlo method,which converts the original quantity of interest into a solution to an optimization problem.After the transformation,the solution process is generally more convenient and efficient.In the field of deep learning,the neural network is generally used as a fitting function.At this time,the traditional process of optimizing the target functional in a specific function space can be transformed into a process of learning neural network parameters,and it is incorporated into the optimization framework of the neural network.This greatly simplifies the optimization process,enabling neural network-based variational methods to be applied to a wider range of more complex problems.This paper first introduces a class of probabilistic generative model based on undirected graph with special structure,namely Deep Boltzmann machine,its principle is expounded,and a new shape completion algorithm is proposed according to its characteristics.By setting the appropriate mask and sampling from the Deep Boltzmann machine,the proposed method can deal with the task without the prior information of the missing region.Then we introduce a new kind of probabilistic generative model,namely the Neural Autoregressive Distribution Esitmator,which is inspired by the Restricted Boltzmann Machine.Combining this model with the mean field method in the Deep Boltzmann machine training process,a better variational learning algorithm is proposed.Experiments show that the model trained with this algorithm has better performance than the original Deep Boltzmann machine.Sequential Variational AutoEncoder is another important probabilistic gen-erative model.In this paper,the self-attention mechanism is introduced into the sequential Variational AutoEncoder,and an integrated framework is proposed,which is applied to the text processing task,so that the model can deal with text classification problems and text generation problems at the same time.By explicitly introducing label into the decoder,the model can generate different kinds of text based on the category information.In addition,this paper explores the latent space of the sequential Variational AutoEncoder.Firstly,it uses the importance sampling to improve the original variational lower bound,which makes the lower bound more tight,and makes the variational posterior distribution closer to the real posterior distribution.Then we introduce the normalizing flow method into the sequential Variational Au-toEncoder.The variational posterior distribution based on the normalizing flow can better fit the posterior distribution and improve the flexibility of the latent space corresponding to the posterior distribution.The experimental results show that,through the adjustment of the hyper parameters,the sequential Variational AutoEncoder based on the normalizing flow can obtain three different genera-tion patterns,which provide the unified framework with the original sequential Variational AutoEncoder and importance weighted sequential Variational Au-toEncoder.At the same time,with the help of normalizing flow,the structure of the latent space has also been expanded.