Deep Generative Models: Learning Algorithms And Applications In Recommendation Systems

Generative modeling is an important unsupervised learning method in machine learning,which can describe the latent features in data in the form of probability distribution.In the era of big data,deep learning methods have achieved remarkable success in scenarios with complex data such as natural images.Deep generative models combine the representational power of deep learning with the statistical foundations of generative modeling,and enjoy many advantages such as the ability to capture the probability distribution of complex data,the ability to answer inference questions in complex data and the generalization ability.Due to these advantages,it has attracted a lot of research attention over the past few years.However,since deep generative models usually consist of complicated neural network structures,their learning and inference methods face many challenges such as: How to assess the model uncertainty of deep generative models;How to introduce prior knowledge for deep generative models;How to prevent deep generative models from overfitting;How to quickly infer the latent variables of deep generative models;How to learn implicit deep generative models without explicit likelihood functions;How to solve the mode collapse problem in implicit deep generative models.To address the above challenges,this dissertation presents a systematic study of learning algorithms for deep generative models and explores the value of its applications in recommendation systems.The main contributions are summarized as follows.1.We propose Doubly Stochastic Gradient Markov Chain Monte Carlo(DSGMCMC),a simple and generic algorithm for(approximate)Bayesian inference of explicit deep generative models in the parameter space.While estimating the gradient of logposterior with respect to the model parameters,the algorithm further efficiently estimates the intractable expectation over hidden variables via a neural adaptive importance sampler.The proposed algorithm manages to improve the learning quality on different explicit deep generative models.2.We present a Learning by Teaching(LBT)approach to learning implicit deep generative models,which intrinsically avoids the mode collapse problem by optimizing a new training criterion which is obtained through theoretical analysis.In LBT,an auxiliary explicit model is introduced to fit the distribution defined by the implicit model while the later one teaches the explicit model to match the data distribution.The training criterion is formulated as a bilevel optimization problem,which is approximately solved by a gradient unrolling technique.The proposed algorithm demonstrates a strong ability to avoid mode collapse.3.We propose User-Item Co-Autoregressive Models(CF-UICA)for collaborative filtering tasks,a neural autoregressive deep generative model which exploits the structural correlation between users and items to describe the generating process of user behavior.Furthermore,we develop an efficient stochastic learning algorithm to handle large scale datasets,overcoming the difficulty that the user behavior data are not independent and identically distributed.The proposed approach achieves superior performance on different recommendation tasks.