Dimensionality Reduction Based On Sparse Representation

With the development of technology,there are large amount of data with high dimensionality in many fields,and such data bring great challenge in storing and computing.An effective method to solve such problems is dimensionality reduction.Due to the difficulty and cost of labeling,the common situation is that the unlabeled data are far more than the labeled data,so unsupervised feature selection and semi-supervised feature selection have become hot researches in machine learning.Dimensionality reduction can be fulfilled by feature extraction and feature selection.Recently the combination of feature selection and subspace learning becomes more and more popular,the characteristic of which is to obtain the transformational matrix by preserving the structural information of data,and then feature selection is guided by the norm of the row or column of transformational matrix.Discriminative features can be selected by preserving the manifold structure of the data.The commonly used structural preserving methods consist of Local Linear Embedding?LLE?and Sparse Preserving Projection?SPP?,which are widely used due to the ability of preserving the manifold structure of the data.However,there are still some problems in such feature selection methods,such as the difficulty of selection of parameters,non-robustness to noise and inefficiency of acquiring adequate discriminative information and so on.Therefore,this thesis deeply studies some feature selection methods,which are based on structural preserving,and proposes some corresponding algorithms.Also extensive experiments are conducted on some standard datasets to verify the effectiveness of the proposed methods.The main research work and innovations of this paper are described as follows:1.Low Rank Sparse Preserve Projection?LRSPP?method lacks the measurement of information between the original data space and the learned subspace that is spanned by the selected features,which causes the loss of information in process of dimensionality reduction.It also lacks the sparse regularization of transformational matrix,which may cause failure in selecting sparse features.Therefore,this thesis proposes a method,global and intrinsic geometric structure embedding for unsupervised feature selection?GGEFS?,that considers the information discrepancy between the original feature space and the lower dimensional subspace,structural embedding and sparse regularization of transformational matrix,where the measurement of information discrepancy can reduce the information loss in the process of dimensionality reduction.The structural preserving is to embed the low rank sparse representation into a lower dimensional subspace,thus preserves the global and intrinsic geometric structure of the data,and the representation is learned from the original data space.We usel_2,12-matrix norm for the projection matrix,which selects more sparse and discriminative features.2.About the semi-supervised feature selection method,this thesis proposes a semi-supervised feature selection method based on low rank sparse graph embedding?SFS-LRSE?.We make full use of all the data in the process of structure learning,the low rank sparse representation of labeled data and unlabeled data are separately computed,and the labeled data are used to learn the structure by their labels.Then the obtained overall low rank sparse graph is embedded into the process of dimensionality reduction,thus the structural information of the data can be sufficiently preserved in the lower dimensional subspace.