Manifold Learning And Sparse Representation With Their Applica- Tions In Pattern Recognition

With the rapid development of modern information technology, people in daily life can easily obtain the required data and information through a variety of sensing de: vices and computer networks. Effectively deal with large-scale high-dimensional data becomes the main problem of the moment that needs to be solved. The research re-sults of cognitive science and neurobiology show that manifold learning methods and sparse representation methods have potential research value and great application val-ue in dealing with high-dimensional large-scale data problems. Based on the manifold learning theory and sparse representation theory which are the forefront of direction in the field of pattern recognition, this study carries out a number of researches on high-dimensional manifold learning, multi-manifold modeling and sparse representa-tion with the manifold assumption to deal with the problem of dimensionality reduction and pattern classification. The main contributions and innovations are as follows:(1)Based on the manifold assumption, we extend the non-parametric discrimi-nant analysis (NDA) method to a semi-supervised dimensionality reduction technique, called Semi-supervised Nonparametric Discriminant Analysis (SNDA). SNDA can take full advantage of the unlabeled sample data and the labeled sample data. There-fore, SNDA overcomes the small sample size problem in the traditional NDA algorithm to some extent. SNDA takes advantage of both the discriminating power provided by the NDA method and the locality-preserving power provided by the manifold learn-ing. SNDA preserves the inherent advantages of NDA, that is, relaxing the Gaussian assumption required for the traditional LDA-based methods. Comparing with the SDA method which is an expanding semi-supervised method based on LDA, SNDA has a better coupling with the nonlinear manifold structure and is more suitable for obvious nonlinear data processing.(2)In the context of semi-supervised learning, we clearly introduced the hybrid manifold classification problem. The hybrid manifold classification assumes that the observed data are sampled from multiple complicated manifolds. Some of these mani-folds have linear structures and some of them have nonlinear structures; some of these manifolds are well separated, while others are intersected. We analyze the potential problems with the existing semi-supervised multi-manifold learning algorithm in the task of classifying hybrid manifolds and point out the issues that may cause perfor-mance degradation in practical applications. And then we propose an effective semi-supervised method to classify the data points sampled from multiple separated or in-tersecting nonlinear manifolds that are embedded in high-dimensional ambient space, called Multi-Manifold Semi-Supervised Gaussian Mixture Model (M2SGMM). In or-der to capture the geometric structure information from multiple hybrid manifolds, we introduce a novel geometrical similarity function based on local tangent space and prin-ciple angles. To detect the hybrid nonlinear manifold structures, we construct a nov-el similarity graph with local and geometrical consistency properties. This enhanced graph is sparse and could be further employed by other graph-based SSL algorithm-s. Additionally, our model is naturally inductive and able to handle the out-of-sample points. We compare our M2SGMM method with five popular semi-supervised learning methods. Experimental results have validated the effectiveness of our method.(3)In the context of unsupervised dimensionality reduction, we propose a Sub-Manifold Sparsity Preserving Projections (SMSPP) algorithm to reduce the computa-tional burden in the construction of sparse l1 graph. Traditional l1 graph based methods use all the training samples to construct the redundant dictionary without taking into account the local manifold structure of the data and usually result in high computation-al complexity. SMSPP prunes the redundant dictionary according to the local manifold structure of the data, so that the atoms in the dictionary contain the manifold discrimi-nant information as much as possible. SMSPP makes a balance between the computing time for the construction of l1 graph and the suboptimal recognition performance by adjusting the size of the dictionary capacity. Experimental results show that the SMSP-P algorithm can obtain high recognition performance while reducing the computation time significantly.