Research Of Restricted Boltzmann Machines Based On Smooth L₀ Norm And Its Application

As a new multi-layer neural networks model with strong representation ability of complex data,deep learning networks model has attracted wide attention of scholars in different fields.In the mainstream models of deep learning,Restricted Boltzmann Machine is often an important component of these models.By optimizing several stacked RMBs,deep networks can efficiently complete training and achieve satisfactory results in the target tasks.This fully shows that RBM plays an important role in deep learning.Therefore,the research on RBM has important theoretical value and practical significance.This thesis firstly improves a kind of RBM and then the improved RBM is stacked to form a deep networks.Finally,the networks are applied to SAR image change detection.Specific research work is as follows.?1?RBM without constraints on the hiddenn layers may produce redundant codes and unstructured weight patterns,which impairs the feature extraction ability of RBM.Sparse representation is able to obtain succinct codes and structured weight patterns,which improve the feature extraction ability of RBM.Adding smooth L₀ norm after objective function is an important method to realize sparse representation,therefore,we propose a novel sparse RBM model based on smooth L₀ norm,referred to as SmoothRBM.We do some experiments on handwritten data set,including visual evaluation of feature extraction performance,experimental analysis of feature sparsity and discriminative ability of representations by improved model.Finally,compared with Weihgtdecay RBM,SparseRBM and PCA models,SmoothRBM has better feature extraction performance than those.?2?A new sparse constrained deep belief networks based on smooth L₀ norm is constructed by stacking several SmoothRBMs,referred to as SmoothDBN.The experiment result shows that classification error rate of SmoothDBN on the test set of handwritten data set is 0.98%,The error rate of some excellent classifiers on the test set,such as DBN,WeihgtdecayDBN and SpaseDBN,are more than 1%.?3?SmoothDBN is applied to SAR image change detection.The main idea of this method is that SmoothDBN directly acting on the two images of the same object at the different time.The results of real SAR images show that the new method has better detection effect than DBN and GKI in terms of OE,Kappa and other evaluation indicators.