Gaussian Distribution Restricted Boltzmann Machines And Its High Dimension Extension

Restricted Boltzmann Machine(RBM)is a special neural network model,which is often used to construct deep neural network.It is widely used in data reconstruction,data classification and pattern recognition.The traditional RBM has only two layers,namely,visible and hidden layers.The elements in these two layers are always in vector form with binary value based on Bernoulli distribution hypothesis.The traditional model is effective on binary data following the Bernoulli distribution,but it will bring some problems on the data distributed in real domain.As the data in reality are mostly complex,so it has great practical value that designing an RBM model which can describe the reality data's distribution better.In addition,the traditional RBM model requires input and output data in vectors.However,the actual data are often in high-dimensional,such as images,video data and so on.When applying RBM on complex high-dimensional data,we must vectorize the structured data firstly,and then utilize RBM model to analysis and representation.The vectorization process destroys the spatial relationship information of the original data heavily and brings dimension disaster.In order to solve the above two problems,this thesis proposed a Restricted Boltzmann Machine model.In this model,visible and hidden layers are based on matrix variables following Gaussian distribution.The model requires that the input and output data have matrix representation,so as to avoid the various problems brought by vectorization process.At the same time,this model can be directly extended to highorder tensor,therefore it can be adapt in a greater application.The work of this thesis mainly includes the following aspects:(1)Aiming at the assumption that visible and hidden units following Bernoulli distribution in traditional restricted Boltzmann machine model,the thesis proposed a restricted Boltzmann machine model with input and output data following Gaussian distribution and then we give the solving methods and training process of the model.The hypothesis of Gaussian distribution allows us to model data distributed over the real domain,therefore it has a wider range of applicability.Reconstruction and classification experiments on multiple public databases demonstrate the effectiveness of this model.(2)Aiming at the problems brought by vector representation of visual and hidden layers,a matrix variable Gaussian distribution restricted Boltzmann machine(MVGRBM)is proposed.And this thesis gives solving methods and training process of the model.Compared with vectorization,this model avoids the problem of destroying the spatial structure information of original data.In order to reduce the model parameters and complexity of the training model,a kind of decomposition representation of the weight tensor is introduced.Reconstruction and classification experiments on multiple public databases have good reconstruction effect and classification accuracy.(3)The matrix variable Gaussian distribution restricted Boltzmann machine is extended to the multimodal data.Thus a multimodal matrix variable Gaussian distribution restricted Boltzmann machine is proposed.The image super-resolution experiment based on this model is also obtained many satisfactory results.