Deep Generative Models For Various Learning Tasks

Deep neural networks(DNNs)have achieved remarkable progress in various machine learning areas and been applied to many real-world scenarios.However,DNNs focus on learning a conditional distribution of a semantic output given an input,which leads to two intrinsic problems.On one hand,DNNs do not model the uncertainty in the input,making it hard to deal with noise and missing values in the input.On the other hand,learning DNNs significantly relies on large amounts of data with semantic labels,which can be rare and expensive in practice.Deep generative models(DGMs)conjoin the flexibility of DNNs and the inference power of probabilistic modeling and can model the underlying distribution of the input data without any semantic label.Therefore,DGMs provide a principle way to solve the two problems of DNNs and will be a basic tool for big data analysis.Though promising,there exist several critical problems of DGMs to be addressed,including the insufficient expressive power,the limited interpretability,and the weak discriminative ability.Firstly,the neural network architectures and latent space representations in DGMs are simple and hence the expressive power of DGMs is restricted.Secondly,current DGMs often learn a black-box mapping from random noise to highdimensional data,such as an image.The process is hard to interpret and control.Thirdly,the features extracted in purely unsupervised learning is less discriminative than that of the feed forward networks,and incorporating the supervision in DGMs properly is nontrivial.This thesis addresses these problems by designing compatible models and learning criteria,and developing effective inference and learning algorithms,in various learning tasks.The novel contributions are summarized as follows.1.In unsupervised learning,inspired by neuroscience,a new building block for DGMs with memory and attention mechanism is proposed to enhance the expressive power of DGMs.An adversarial variational inference and learning algorithm is proposed to train a large family of undirected graphical models effectively without any specific assumption about the model architecture.2.In unsupervised and weakly-supervised learning,an implicit model with arbitrary directed structural prior and an adversarial expectation propagation algorithm is proposed as a general and flexible framework,which can infer the latent structure of the input and synthesize structural samples,to enhance the expressiveness of the latent space and the interpretability of DGMs.3.In supervised learning and semi-supervised learning,a max-margin based learning criterion is proposed to improve the discriminative ability of explicit models while retaining the generative power.4.In semi-supervised learning,a game-theoretical implicit model is proposed to boost the classification results significantly and control the semantics of the generated data for the first time.