Font Size: a A A

Extensible And Scalable Bayesian Learning Methods: Modeling And Inference

Posted on:2016-10-16Degree:DoctorType:Dissertation
Country:ChinaCandidate:M J XuFull Text:PDF
GTID:1108330503956151Subject:Computer Science and Technology
Abstract/Summary:PDF Full Text Request
Many popular and useful machine learning methods, e.g. support vector machines and matrix factorization methods, are based on solving for an optimal model that minimizes its regularized empirical risks. On the other hand, Bayesian learning methods tackle the distribution of the model. They enjoy great flexibility and robustness as well as a wide range of developed inference algorithms. Hence we would like to transform some of those regularized risk minimization problems into Bayesian learning problems so as to exploit the rich tools therein for both modeling and inference to obtain even better learning methods. Meanwhile, the prosperity of the Internet brings along several new kinds of data, e.g.relational data, extending Bayesian methods to analyze which becomes a worthy problem for research. The advent of the Big Data era also raises the standard for the efficiency and scalability of Bayesian inference algorithms.In this paper we first investigated how to transform a regularized risk minimization problem into a Bayesian learning one. We closely compared two different approaches to this end, namely traditional Bayes-rule based methods and maximum entropy discrimination methods, and demonstrated several intrinsic differences and connections between them. Then as a concrete example for the application of such transformation approaches,we focused ourselves on the maximum margin matrix factorization(M3F) method that is used for analyzing “user-item” rating data. We introduced nonparametric Bayesian methods into the resulting Bayesian model and successfully handled the problem of selecting the number of the latent factors. Finally we proposed a general distributed sampling algorithm for Bayesian posterior simulation and greatly improved the scalability of Bayesian posterior sampling algorithms in the big data setting.Here we summarize the major contributions and novelties of this paper.1. We obtained, through layered abstraction and a generalizable formalization, a general and direct paradigm to transform a regularized risk minimization problem into a Bayesian learning problem.2. We introduced nonparametric Bayesian methods into maximum margin matrix factorization, automatically inferring the number of latent factors during the learning process and, at the same time, improving the prediction performances. We borrowed a specific data augmentation technique designed for support vector machinesinto our Bayesian learning methods of M3 F and obtained an efficient Gibbs sampling algorithm for approximate inference.3. We proposed a novel distributed Bayesian posterior sampling algorithm. Unlike previous approaches that are “embarrassingly parallel”, we allow computing nodes to share moments of their local posteriors with parsimonious and infrequent message passing. At convergence, all the participating nodes would obtain accurate samples from the global posterior.
Keywords/Search Tags:machine learning, Bayesian, matrix factorization, sampling
PDF Full Text Request
Related items