Research On Unsupervised Feature Learning Algorithms Based On Sparse Modeling And Information Theory Learning

With the improvement of data collection technology,the dimension of the data samples are growing higher and higher.The high dimensional data not only contain more useful information,but also always introduce much redundancy and increase the computational complexity of learning algorithms.In order to adapt the increasing of dimension of data,process the data efficiently and decrease the redundancy of the data,how to efficiently learn the low-dimensional representation becomes an urgent problem in nowadays big data processing.In reality,data is much easier to be obtained than be labeled.It is very consuming,no matter time or money,to label the data,especially the number of data is very large.Therefore,unsupervised feature learning methods become a hot topic nowadays.The research topic of this paper is the unsupervised feature learning algorithms,and it focus on the algorithms which is based on the sparse modeling and information theory learning.Specifically,the topic of this paper can be divided into two parts: unsupervised feature selection algorithms and information theory learning based unsupervised feature learning algorithms.Firstly,the sparse modeling based unsupervised feature selection algorithms are studied.The unsupervised feature selection problem is formulated as the subspace learning model.There are three subspace learning based feature selection models are proposed:1)In order to capture more useful information and eliminate the negative contribution,a nonnegative subspace learning model is proposed.To more efficiently dig information in the data,this paper introduce the adaptive sparsity framework into subspace learning process and propose a nonnegative sparsity adaptive sparse subspace learning model for unsupervised feature selection;2)In order to utilize the intrinsic information of data,the local structure preserving term is incorporated into the subspace learning model and propose a global and local structure preserving sparse subspace learning method;3)In order to add the discriminative information in unsupervised learning scenario,the clustering assignment information be utilized as discriminative information to guide the subspace learning process and propose a discriminative sparse subspace learning method.Secondly,the information theory learning based robust version of unsupervised feature learning methods are studied.If the data contain outliers,the performance of the feature learning method will degenerate,if the objective function of model is formulated by the Frobenius norm.Because of the sound robustness of the lost functions in the information theory learning,the maximum correntropy criterion(MCC)is used to formulate the objective function and propose two MCC based robust feature learning model: 1)In order to improve the outlier robustness of unsupervised feature selection model,the MCC is used to combine with local structure preserving subspace learning term and propose the MCC based robust unsupervised feature selection model;2)In order to improve the robustness of SPCA model,the MCC is used to model the lost function.Furthermore,in order to dig the intrinsic information of data,the multi-hypergraph regularization term is added into the SPCA model and propose a MCC based robust high-order manifold constraint SPCA method.