The Study Of The Manifold Learning Based Supervised Dimensionality Reduction Methods

With the technology development, a large amount of high dimensional data emerged which always causes the classifier malfunction and needs large computation. Those data always have nonlinear distributions, which make the classical dimensionality reduction method hardly detect its structure effectively. Based on the assumption that local linearity and global nonlinear, manifold learning algorithms can detects the intrinsic nonlinear structure of the dataset. Therefore, manifold learning is a hot topic in dimensionality reduction and feature extraction field. In this thesis, manifold learning methods are taken used to reduce the dimension of the data set. However, the classical manifold learning methods are merely based on dataset and without considering the class information of the dataset. In this thesis we mainly focus on taking good use of the label information in the manifold learning. In order to meet this requirement, we designed two supervised algorithms based on manifold learning. A better prediction result acquired, through employing those algorithms on face dataset and cancer dataset.The main work of this thesis can be summarized as follows:(1) Feature selection and feature extraction are introduced. Feature selection methods select a subset of features from the given set of features. Feature extraction methods determine an appropriate subspace of dimensionality based on the original feature space of dimensionality. Also, the classical manifold learning methods are introduced.(2) Three classical supervised manifold learning based dimensionality reduction methods introduced, such as Discriminant Local Linear Embedding (DLLE), Local Sensitive Discriminant Analysis (LSDA), A Freamwork of Greap Embedding and Marginal Fisher Analysis (MFA). The Linearization, Kernalization, Tensorization of those algorithms is also introduced. In this part the connection between Graph embedding algorithm and other classical dimension reduction algorithm have beed introducted as well.(3) Avoiding the defective of the prevailing manifold learning algorithm, we proposed a new algorithm named Orthogonal Discriminate Projection. This algorithm takes consideration of the class information and the distribution information at the same time. Based on this information, the different distance functions are defined. The experiment result on face dataset, cancer dataset displays the effectiveness of the proposed algorithm.(4) An algorithm named Local Sensitive Frontier Analysis has been proposed. Inlighted by the SVM algorithm, which denote the Support Vectors are the instance close to the classification hyperline in the high dimensionality. We denote the sensitive instance that close to other classes in the original space. According to their sensitivity different punishment weights are given. After projection the samples are more convenient to classification. And the experiment results display the efficience of this algorithm.