Algorithms Of Feature Extraction Based On Deep Auto-Encoders

Machine learning plays an extremely important role in the field of artificial intelligence,it allows the machine to be able to build a model for a variety of things.These models give machine the ability of “understanding” our world.In recent years,machine learning has been rapid developed.on the one hand,thanks to the development of computer hardware technology,it significantly reduce the cost of storage,and computing speeds are significantly improved.More importantly,research on machine learning algorithms obtained a breakthrough,especially in recent years proposed deep learning algorithms,making the machine cognitive ability to the world has a qualitative leap.Deep Auto-Encoders is one of deep learning algorithms,which is a deep neural network.Its biggest characteristic is through a layer-wise unsupervised pre-training makes the network a good initial value,and then through supervised fine-tuning training adjust the entire weight of the network.Ultimately,it can effectively extract the key information in the data and form features.Based on the research Deep Auto-Encoders,we also pay attention to study types of commonly used feature extraction algorithms and classification algorithms.The experiments prove the differences in the various algorithms,and we explore ways to improve the Deep Auto-Encoders.The main contents are as follows:(1)Many kind of feature extraction algorithms have been studied.Including the classic feature extraction algorithm: Principal Component Analysis,Linear Discriminant Analysis,Kernel Principal Component Analysis.In order to distinguish deep learning algorithm,these algorithms are summed up as “shallow” learning algorithm.Except the Deep Auto-Encoders,the deep learning algorithm based on Restricted Boltzmann Machine and several improvements of Deep Auto-Encoders are also been studied.Image recognition experiments have been made to compare the performance of each algorithm.(2)Various classifiers have been studied,including Softmax,Support Vector Machines and K nearest neighbor.Experiments on Deep Auto-Encoders combining different classifier have been made to compare the classification performance.(3)To further enhance the features learning ability of Deep Auto-Encoders,a marginalfisher analysis algorithm based on stacked Auto-Encoders has been proposed which can further improve the ability of representation learning by applying marginal fisher analysis to the supervised fine-tuning phase.It has the intrinsic graph which describes the intra-class compactness and the penalty graph which describes inter-class separability.With these two graphs,we can optimize the way that data mapping from original space to feature space.The method feasibility verified by experiment.