Font Size: a A A

Research On Deep Neural Networks Based On RBM Or ELM-AE

Posted on:2020-03-08Degree:DoctorType:Dissertation
Country:ChinaCandidate:N ZhangFull Text:PDF
GTID:1368330590951854Subject:Computer application technology
Abstract/Summary:PDF Full Text Request
The restricted Boltzmann machine(RBM)is a kind of generative network.It can not only learn the representations of the data,but also reconstruct the data by using the representations.The extreme learning machine as auto-encoder(ELM-AE)is also a network for data encoding,but it does not need iteration.Both of them can be used for representational learning and learning the features of the data,so they can be both used as the basic building blocks for creating deep networks.At present,the research on deep neural networks based on RBM or ELM-AE has made many achievements,but there are still many problems worthy of further study.This thesis mainly focuses onthe the application of deep networks on noise data classification,image denoising,multi-view data classification,semi-supervised learning and multi-label learning.The specific research contents are as follows:1.Study point-wise gated deep networks based on clean data and noise data.Traditional deep networks based on the RBM don't show better learning abilities than the RBM when dealing with data containing irrelevant patterns.Assume that irrelevant patterns in data lead to bad intermediate level representations and affect the performance of deep networks,this thesis combines point-wise gated RBM(pgRBM)with deep networks and presents point-wise gated deep networks,where the feature selection strategy is used to enhance the classification abilities of deep networks.Given that train data is composed of noisy data and clean data,this thesis proposes pgRBMs based on noisy data and clean data(pgncRBM),which use clean data to obtain the initial values of the task-relevant weights and achieve better classification results on noisy data.And then,this thesis stacks pgncRBM and RBMs to create deep networks for noisy data classification.2.Study robust spike-and-slab deep Boltzmann machines for image impainting.The Robust Gaussian RBM(RoGRBM)achieves a better result in the face denoising task,and it make use of a Gaussian RBM to model the clean data.The Spike-and-Slab RBM shows better learning abilities than the Gaussian RBM in real images modeling.In addition,the Deep Boltzmann Machine(DBM)shows powerful image reconstruction ability.This thesis first stacks the Spike-and-Slab RBM and the RBM to create the Spike-and-Slab DBM for modeling the real images.And then,this thesis utilizes the Spike-and-Slab DBM instead of the Gaussian RBM to model the density of the clean data in the Robust Gaussian RBM,and the proposed method is named as the Robust Spike-and-Slab DBM.It makes use of the learned Spike-and-Slab DBM model and the mean field method to obtain better de-noising images.3.Study restricted Boltzmann machines for multi-view learning.The general RBM is only suitable for addressing the single view data.This thesis first presents a multi-view RBM model for multi-view classification,named as the RBM with posterior consistency(PCRBM).The PCRBM computes multiple representations by regularizing the marginal likelihood function with the consistency among representations from different views.However,the learned representations should not just contain the consistency information.Then,this thesis presents a new multi-view RBM model and named as the RBM with posterior consistency and domain adaptation(PDRBM).The PDRBM divides the hidden units of a RBM on each separated view in two groups: one group that contains the consistency information among different views and the other group that contains the information unique to this separated view.4.Study multi-layer extreme learning machines for semi-supervised learning or multi-label learning.Multi-ayer extreme learning machines(ML-ELM)can learn the deep representations of the data,and most of traditional semi-supervised learning or multi-label learning algorithms are shallow algorithms.This thesis combined the representation learning of ML-ELM with traditional algorithms to improve the performance of traditional algorithms.This thesis first introduces manifold regularization into the ML-ELM and propose laplacian ML-ELM,which takes the advantages of semi-supervised learning,deep leaning,and the extreme learning machine.Then,this thesis presents a neural network named multi-layer ELM-RBF for multi-label learning(ML-ELM-RBF).ML-ELM-RBF firstly stacks ELM-AE to create a deep network,and then it conducts clustering analysis on samples features of each possible class to compose the last hidden layer.
Keywords/Search Tags:deep learning, representational learning, restricted Boltzmann machine, extreme learning machine as auto-encoder
PDF Full Text Request
Related items