Research On Feature Learning Algorithm Based On Auto-encoder And Its Application

With the rapid development of information technology,deep learning has become a popular research topic.Auto-encoder is a typical model in deep learning,and has also been widely studied.Auto-encoder as an effective algorithm for feature learning can extract the features of the data.In this paper,the features are used for classification which are extracted from the Auto-encoder,and the accuracy of the classification is verified to determine the feature learning ability of the Auto-encoder.The specific work was as follows:(1)Handwriting recognition research is an important branch in the field of character recognition.Because of the obvious characteristics and large differences in edge contour,this paper proposes an algorithm named Jacobian regularized sparse Auto-encoders(JSAE).By adding the sparsity constraint into Auto-encoder makes the improved Auto-encoder extract hidden structure from the data.Through adding the Jacobian regulation into auto-encoder makes the improved auto-encoder describe edge feature of point data,improve the feature learning ability and extract the essential characteristics of the sample more accurately.So adding these two regularized items can improve the accuracy of the handwriting recognition.The algorithm has better classification accuracy on MNIST,USPS and Pen Digits datasets.(2)Due to the high time complexity and the gradient disappeared of the adjusting traditional deep auto-encoder,this paper proposes an algorithm named ELM optimized deep Auto-encoder(DAEELM)which using extreme learning and related algorithms instead of back propagation.Because the extreme learning machine has faster training speed and strong generalization capability,DAEELM can effectively avoid the training time and gradient disappeared.Consider label information and then adjust parameters of network accordingly.Then the adjusted network structure is applied to classification.Finally,ELM is adding as the classifier to the last layer,instead of the traditional Softmax classifier.This way avoids the process of fining tune the whole network,and greatly shortens the training time.The algorithm has better classification accuracy on Pen Digits,USPS and ISOLET data sets.(3)Semi-supervised Auto-encoder is a promising deep learning method which extracts the underlying abstract concepts of data using its unsupervised learning process,and the extracted distinctive information would enhance the generation ability of the structure.Meanwhile the supervised process of SSAE extracts the features which tends to describe the given categories with labels and would further improve the categorization accuracy of the classification models.In this paper,we propose a semi-supervised method,named sparse and label regulation AE(SLRAE),by adding label and sparse regulations.The sparse regulation makes minority neurons activated and the most of other neurons inhibited.Such a method would ensure SLRAE obtain more local and informative structure of the data.Moreover,through consideration of label regulation,the supervised learning can extract division rule-based features to describe the given categorizations.The algorithm has better classification accuracy on 6 UCI data sets,a USPS dataset and a MNIST dataset.