Markov Chain Monte Carlo Based Improvements To The Learning Algorithm Of Restricted Boltzmann Machines

Deep Learning is a frontier research field of machine learning in recent years. Itfocuses on the problem of modeling and learning using neural networks with multiplelayers, such"deep networks"is more similar to human brain in information process-ing, thus it is considered as a much more powerful tool for complex AI problems.The state-of-the-art solution of deep learning is training the deep network as it's astack of multiple RBMs (Restricted Boltzmann Machines). Since RBMs are relativelyeasy to learn, this kind of approach overcomes the efficiency problem of directly train-ing a multiple layered network. So, the learning of RBMs is a fundamental problemin deep learning research.In this thesis, we will further discuss the learning problem of RBMs. Our maincontributions are: (1) We summarized the learning algorithms of RBMs proposed inrecent years, and discovered that MCMC is a promising method for better learningRBMs; (2) We investigated the deficiency of the current solutions—they only usedMCMC as an external tool, without sufficiently taking advantages of the learning pro-cess to enhance MCMC in return; (3) We proposed an idea of combining MCMCwith learning together, and designed a learning specific MCMC algorithm; (4) The ex-periment results shown that, our algorithm is superior to the current methods in bothefficiency and likelihood; (5) We proposed several new metrics including the contentand the convergence of the mode set.As the development of deep learning, MCMC-based machine learning algorithmswill be more and more important. As MCMC and machine learning get more close, thelearning specific MCMC algorithms will arouse many research interests in future. Ourwork firstly propose and evaluate this idea, and demonstrate its potentials, showing apossible way of this new direction.