Research On Deep Learning-Based Representation Learning Algorithms

Representation learning is the method to obtain vectorized representations of entities or relationships with machine learning or data mining algorithms.The aim of representation learning is to represent the semantic information as the dense low-dimensional real-value vectors by the algorithm of machine learning.Professor Yoshua Bengio,a leading expert in machine learning and neural networks,explained the importance of representation learning:The success of machine learning algorithms generally depends on data representation,and we hypothesize that this is because different representations can entangle and hide more or less the different explanatory factors of variation behind the data.Significant improvements to representation learning have emerged in recent years.In the top academic conference and journals of artificial intelligence like CVPR,NIPS,SIGKDD,AAAI and IEEE Transactions on Image Processing,IEEE Transactions on Geoscience and Remote Sensing etc.,a number of papers on representation learning are published.With the development of big data and information technology,how to learn higher-level and more abstract feature representations from data has become an important research problem.Deep learning refers to the algorithm set that solves various problems about images and texts by using various machine learning algorithms on a multi-layer neural network.This paper aims at the problem of how to maintain the independence of data vectors,the distribution similarity of data space,and how to optimize the feature representation with few label information to make the data representation more robust and abstract.In different application areas such as transfer learning or multi-task learning,the representation learning method based on deep sparse self-coding achieved good performance in classification task.We build a unified shared potential feature space and make the data distribution across different domains consistent in this space.However,how to maintain the inherent structure of input data and avoid the requirement of massive labeled data in deep learning architecture is the problem when faced the large image data sets.To adress this problem,we proposed the semi-supervised learning method to construct a representation learning model based on stacked convolution sparse auto-encoder,which has achieved good performance and has important research and application value.We researched on deep learning-based representation learning algorithms,and several representation learning algorithms were proposed.Our main contributions are listed as follows:(1)A novel transfer learning method based on deep learning architecture and high-level data feature representation learning,called SRICA,was proposed.Our work is motivated by the following problems.Firstly,the independence of the vector representation in the data representation and the similarity of the data space need to be maintained,but the traditional deep learning model is difficult to indicate the sparsity of the data effectively.Secondly,due to the label information in the source domain of the transfer learning,the domain discrepancy can be alleviated with the labeled data.To adress these problems,the data is mapped into the shared potential space with the algorithm of reconstruction independent component analysis,the label information in source domain is encoded for optimizing the feature representation,and the KL distance is introduced to minimize the distance between source and target domain.Extensive experiments conducted on several image datasets demonstrate the superiority of our proposed method compared with all competing state-of-the-art methods.(2)A novel multi-task learning method is proposed,which called DSML and learned model parameters and feature representations between multiple tasks simultaneously.Since the common and particular model parameters for each task model in multi-task learning are existed,it is difficult to optimize the common model parameters based on feature representation optimization.To address these problems,the algorithm of deep sparse auto-encoder is proposed to optimize the common parameters of the model,which taken the sparsity of the feature representation into consideration and improved the quality of data representation.In the process of parameter tuning,a more concise model is used,which reduces the computational complexity and improves the efficiency of the algorithm effectively.Extensive experiments on several real image datasets demonstrate our proposed framework outperforms the state-of-the-art methods.(3)A novel representation learning method is proposed,which called SSCAE and optimized the inherent structure of feature representation with convolution way based on semi-supervised learning.The supervised deep learning methods such as deep convolutional neural networks require large amounts of labeled data,and unsupervised deep learning methods such as auto-encoder can not make efficient use of label information.To address these problems,SSCAE proposed the method of deep sparse auto-encoders with whiten layers for representation learning.The softmax regression model is used to optimize the label encoding,and the feature representation is optimized with convolution way to learn higher-level feature representations for tasks such as data classification.Extensive experiments demonstrate the superior classification performance of our framework compared to several state-of-the-art representation learning methods.