Graph Based Semi-supervised Dimensionality Reduction Algorithm Research And Its Application

Dimensionality reduction attracts lots of attention in machine learning. As the technology of obtaining information developing, it brings the curse of dimensionality. How to acquire useful information from the high-dimensional data is an important problem. Since the labels of data are really difficult to collect, while the unlabeled data are abundant and easy to obtain, this dissertation studies some existing dimensionality reduction methods and proposes three new semi-supervised dimensionality reduction methods, which can use both labeled and unlabeled data. The proposed methods are successfully used in face recognition, handwritten digits recognition, and hyperspectral remote sensing image classification.The main contributions can be summarized as follows:(1) A new semi-supervised dimensionality reduction method based on local scaling cut (LSC) criterion is proposed and applied to hyperspectral remote sensing image classification. First construct the dissimilarity matrices based on LSC, then define a regularization term based on the prior knowledge provided by the abundant unlabeled data and add it to LSC. The new low-dimensional space is obtained by solving an eigenvalue problem. In the new space, the dissimilarity between samples in the same class is as lower as possiple, and the dissimilarity between samples in different classes is as higher as possiple. What’s more, if two samples are close in the original space, their projected vectors are desired to be close. Experiments on hyperspectral remote sensing images show that the proposed algorithm significantly improves the accuracy rates of Support Vector Machine (SVM) classfier, and gives a relatively promising performance compared to other traditional dimensionality reduction methods and the corresponding supervised approach.(2) A new semi-supervised dimensionality reduction method based on sparse representation is proposed. The graph constructed by sparse representation can characterizes both the intrinsic geometry of data and considerable discriminative information. We define a regularization term by utilizing the l1-graph based on sparse representation and combine the regularization term with the scatter matrices defined in the traditional linear discrimination analysis (LDA). Experiments on face recognition, handwritten digits recognition, and hyperspectral remote sensing image classification show that the proposed algorithm can achieve better and more stable results.(3) A new semi-supervised discrimination criterion called sparse local scaling cut criterion is proposed and be applied to the dimensionality reduction of hyperspectral remote sensing image. The proposed method firstly calculates the sparse weight matrix of the unlabeled data based on sparse representation, and constructs a regularization term. We define the new criterion by adding the regularization term to LSC. Maximizing the criterion, we will obtain the optimal projective space, which can preserve the separability of labeled samples in the original space and the sparse reconstruction relationship between the unlabeled data. Experiments on hyperspectral remote sensing image classification show that our algorithm gives better results compared to other traditional methods.This work was supported by the National Natural Science Foundation of China (No.60803097), the National Science Basic Research Plan in Shaanxi Province of China (No.2011JQ8020), and the Fundamental Research Funds for the Central Universities (No. K50511020011).