Efficient Deep Learning Algorithm With Accelerating Inference Strategy

Machine learning is an important area of artificial intelligence. In recent years, deeplearning is a new machine learning of training multilayer neural network，which can beused to alleviate the local minimum issue. Deep Learning has provoked great interest inmachine learning community．Deep Boltzmann Machine(DBM) is one of deep learning models, which is based onthe engery and proability. In the pretraining phase, usual bottom-up inference process wasadded to the feedback of top-down pass, which makes DBM better learn uncertainly fuzzyinputs and produce better model of data. However, the approximate inference process thatis a kind of the data-dependent expectation of DBM using the mean-field equations wasmuch time-consuming. The training of DBM would be very slow on large scale dataset. Inthis study, we present an efficient learning algorithm to get the data-dependent expectationquickly for Deep Boltzmann Machine (DBM), using a generative model with multi-layersof variables. This algorithm adopts a layer-wise accelerating inference strategy to computethe mean values of all hidden layers, instead of computing the mean values by repeatedlyrunning the equations of mean-field fixed-point until convergence. It is possible toefficiently learn a good generative model of high-dimensional highly-structured sensoryinput. By taking advantage of layer-wise inference strategy, we can rapidly get theapproximate mean values in a few iterations. This strategy also could learn efficiently ahigh performance model for high-dimensional high-structured sensory inputs. Theproposed algorithm with layer-wise accelerating inference (LAIDBM) performs wellcompared to original Deep Boltzmann Machine, Deep Boltzmann Machine based onrecognition model, Deep Belief Network and shallow network algorithm given learningtasks.