Research On Model Optimization Of Restricted Boltzmann Machine And Deep Representation Learning

Representation learning has long been one of focus on machine learning.The feature representations and their distributions directly determine the performance and efficiency of next tasks.The Restricted Blotzmann Machine(RBM)is a classical representation learning model and it is one of basic modules of deep learning network.It has powerful capability of feature extraction in the representation learning.There are many successful cases in practical applications.However,there are not any auxiliary information to guide the features encoding of traditional RBM in the training process.Furthermore,conventional RBM has not collaborative representation capability.Hence,the auxiliary information is used to guide the encoding process of hidden features during the training procedure of RBM in this dissertation.Then,a series of novel variants and their deep learning framework are proposed for representation learning.The main works are as follows:(1)A pairwise constraints RBM with Gaussian visible units(pc GRBM)model is proposed.The pairwise constraints information are fused into the representation learning process of RBM.Then,a novel semi-supervised variant model which terms pc GRBM is developed.The instances of same cluster in the Must-link set are flocked together and the instances of different clusters in the Cannotlink set are separated as much as possible in the reconstructed instance space of pc GRBM.All experimental results demonstrate that the pc GRBM model has excellent semi-supervised representation capability.(2)Two novel variant models which term Minor Constraint Disturbances RBM(MCDGRBM)and Minor Constraint Disturbances GRBM(MCDGRBM)and a novel Deep Semisupervised Feature Learning framework are developed.Two novel MCDGRBM and MCDRBM models are proposed for semi-supervised feature learning with less labels.The stability of the representation learning in high-dimensional data space is raised simultaneously.Then,a novel Minor Constraint Disturbances-based Deep Semi-supervised Feature Learning framework(MCD-DSFL)is proposed based on these variant models.The probability distributions of hidden layer features of MCD-DSFL become as similar as possible in the same cluster and they are as dissimilar as possible in the different classes simultaneously by fusing the Minor Constraint Disturbances(MCD)in the deep feature learning process.Experimental results show that the MCD-DSFL framework has excellent deep feature semi-supervised representation capability and the MCD has the leverage effect during the process of deep learning.(3)An unsupervised multi-clustering integration RBM(MIRBM)model is presented.The auxiliary information LCP which is generated by multi-clustering integration method is fused into the representation learning of RBM.A novel variant which terms multi-clustering integration RBM(MIRBM)is proposed.In the training process of MIRBM model,the hidden features and reconstructed hidden features of the same cluster in the LCP set gather together as much as possible.At the same time,the center of each local cluster in the LCP set is dispersed as far as possible.Then,the distribution of hidden features are optimized using the auxiliary of LCP.Experimental results prove that the MIRBM model has excellent unsupervised representation learning ability.(4)Two novel variant models which term collaborative representation RBM(cr RBM)and collaborative representation GRBM(cr GRBM)and a novel unsupervised collaborative representation deep network(UCRDNet)are proposed.The local minor data block is generated by LSH method.Then,the collaborative relationship of instance and feature of these minor data block are fused into the hidden encoding procedure of RBM.Hence,two novel collaborative representation RBMs which term cr RBM and cr GRBM models are developed.In the training process of the variant models,each minor data block in the hidden feature and reconstructed hidden feature spaces are is as close to each center as possible.A novel Unsupervised Cooperative Representation Deep Network(UCRDNet)is constructed based on cr RBM and cr GRBM models.Experimental results demonstrate that the UCRDNet has superior performance of deep collaborative representation using the auxiliary of the collaborative relationship of these minor data block.