The Improvement Of Restricted Boltzmann Machine And Its Application

Deep learning has attracted widespread attention in the field of machine learning as a way for learning the distributed features of data.Among the deep learning approaches,deep belief network(DBN)is one of the most popular methods,and has been successfully applied to various machine learning tasks.Restricted Boltzmann machine(RBM)is a generation model that is able to automatically extract data features in a unsupervised learning manner.At present,RBM has been widely used in many fields because of its powerful ability of feature extraction and the basic structure module as the deep confidence network,which has attracted the close attention of the machine learning community.In this paper,based on the analysis the learning algorithm of restricted Boltzmann machine,that is,contrastive divergence(CD),we show that the RBM may have the problem of feature homogeneity.In response to this problem,combined with sparse representation,this paper attempts to improve the Boltzmann machine.1.Combined with sparse representation,in this paper we use arctan function to encourage hidden units to be sparse and propose a novel sparse restricted Boltzmann machine,referred to as AtanRBM.AtanRBM applies the arctan function on the totality of hidden units' activation probabilities to achieve sparse representations,so as to avoid the emergence of feature homogeneity problems.In this approach,the level of sparsity corresponding to each hidden unit can be automatically learnt based on the task at hand.Experiments in the MNIST dataset show that AtanRBM can learn more sparse and more discernible features than RBM and Sparse Restricted Boltzmann Machine(SRBM).Meanwhile,AtanRBM can be used to pre-train deep belief network,and the deep belief network can achieve better classification performance on the MNIST dataset.2.Due to the possibility of statistical correlation between hidden units,we propose a new RBM model based on elastic network,referred to as EN-RBM.Among them,we adopt an appropriate strategy to introduce the elastic network penalty into the logarithmic likelihood function of RBM.This strategy divides the hidden units into two groups by using the similarity of features of hidden units of RBM,and uses the1 L norm and2 L norm to constrain the activation probability of the corresponding hidden units respectively,in order to ensure the sparseness of the representation and the generalization capabilities.Experiments in the MNIST dataset show that EN-RBM has a stronger unsupervised learning ability.At the same time,EN-RBM can be used to build a deep belief network,and the deepbelief network can better complete the classification task on the MNIST dataset.