Research On Manifold Learning And Its Applications

As a novel nonlinear dimensionality reduction technology, manifold learning has become one of the most active research areas in machine learning, data mining and pattern recognition. This thesis has made research systematically and thoroughly on performance improving techniques of classical manifold learning algorithms. The main work and contributions can be summarized as follows:1. Proposing an outlier detection method for robust manifold learning. Traditional outlier detection methods, which have not considered the manifold assumption of local linear and global nonlinear, are not suitable for manifold learning. To improve the robustness of existing manifold learning algorithms, this thesis presents a novel iterative outlier detection method based on coding length. The coding length, a structural descriptor, can characterize the geometric structure better than the traditional Euclidean distance. The use of iterative shceme can make the proposed method more topologically stable than the traditional methods. The proposed outlier detection method can serve as a preprocessing procedure for many classical manifold learning algorithms and make them more robust.2. Proposing a neighborhood selection method in presence of noisy data. Considering that the neighbors based on Euclidean may not be that in the sense of manifold, this thesis presents a new neighborhood similarity measure called neighbor ranking metric (NRM). Based on NRM, a new neighborhood selection method called NS-NRM is also presented. The basic idea of NS-NRM method is to extend each neighborhood while removing its non-neighbor elements. Compared with traditional kNN method, the NS-NRM method is not sensitive to the parameter and more suitable for the noisy manifold learning.3. Proposing a modified local tangent space alignment algorithm to overcome the shortcomings of the original algorithm in dealing with sparse or non-uniformly distributed data. At first, a new L1 norm based weighted PCA method is presented to estimate the local tangent space of the data manifold. By considering both distance and structure factors, the proposed method is more accurate than the traditional PCA method. To reduce the bias of coordinate alignment, a weighted scheme based on manifold structure is then designed, and the detailed solving method is also presented. Experimental results on both synthetic and real datasets demonstrate the effectiveness of the proposed algorithm when dealing with sparse and non-uniformly manifold data.4. Combining manifold learning with the sparse representation theory and the nonparametric discriminant analysis technique, two supervised feature extraction methods called SPPNDA and SRNDA are proposed. SPPNDA seeks for the optimal projection matrix by simultaneously maximizing the nonparametric between-class scatter and preserving the within-class sparse reconstructive relationship. SRNDA seeks for the optimal projection matrix by simultaneously maximizing the nonparametric between-class scatter and minimizing the sparse local scatter. The characteristics of SPPNDA and SRNDA are listed as follows:first, they construct graph based on sparse representation, avoiding the difficulty of parameter selection encountered in kNN method. Second, they use the nonparametric technique to characterize the between-class information, which allows them to work well for non-Gaussian distributed datasets.5. Proposing a feature selection algorithm based on subclass structure preserving. The existing manifold learning-based feature selection algorithms, such as Laplacian Score (LS) and Minimum-maximum Local Structure Information Laplacian Score (MMLS), are often sensitive to parameters and not suitable to deal with complex distributed datasets. To perfect the imperfection mentioned above, this thesis proposes a new manifold learning-based feature selection algorithm based on subclass structure preserving, SSPFS in short. Firstly, the SSPFS introduces the affinity propagation method for subclass division. Then, the graph model is exploied for within-subclass structure and between-subclass structure description. Finally, the SSPFS selects features by simultaneously maximizing the between-subclass information and minimizing the within-subclass information. Since the local geometric structure and class information are involved in the subclass structure information, the proposed algorithm can perform well in the practical applications, as observed from the experimental results on face and remote sensing datasets in this thesis.