Deep Learning Algorithm Based On The Sparse Auto-Encoder And Marginalized Denoising Auto-Encoder

In recent years,deep learning has become a popular research direction in the field of machine learning.Deep learning model could simulate the information processing mechanism of human being brain.As an effective deep learning model,Auto-Encoder could be used to extract the feature of data.The deep neural network that consist of Auto-Encoder has excellent learning performance.In this thesis,we present a deep learning neural network model which combined the constraint condition of Sparse Auto-Encoder(SAE)and Marginalized denoising Auto-Encoder(mDAE).Sparse Auto-Encoder requires the hidden layer neurons to be sparsity,so as to extract useful feature for the input data with a small number of hidden layer neurons.Marginalized denoising Auto-Encoder's key idea is to marginalize out the noise of the corrupted inputs in the Denoising Auto-Encoder,so as to learn robust representation and make a few more computationally convenient simplifications.We present an improve Auto-Encoder algorithm that combined the sparsity constraint condition of SAE and the marginalized noise of the corrupted inputs constraint condition of mDAE to form Sparse Marginalized denoising Auto-Encoder(SmDAE).SmDAE is an Auto-Encoder which uses the constraint conditions of SAE and mDAE.SmDAE has an advantage over SAE because SmDAE has robustness to the noise of corrupted inputs and could eliminate the influence of the noise of the corrupted inputs during the training.SmDAE also has an advantage over mDAE because SmDAE's hidden layer neurons was sparse and could extract more useful feature with a small number of hidden layer neurons.Experimental results show that SmDAE outperforms both SAE and mDAE in the given experimental datasets.