Manifold Learning And Its Applications In Pattern Recognition

With the fantastic development of information technology, data collection has become increasingly easy. However, the large amounts of data resources have puzzled people, due to their huge quantity, high dimensionality and nonlinearity. Although the data resource is sufficient, we are confronting with the embarrassment that the needed information cannot be discovered. And then, a new research direction of high dimensional data analysis, called, manifold learning, emerged as the times require, and attracts a surge of research interests. The goal of manifold learning is to solve the difficulties caused from the nonlinearity of data distribution in high dimensional dataset, and to explore the faithful intrinsic geometry hiding in high dimensional dataset.In this dissertation, we expand manifold learning towards to pattern recognition and the motivation is to facilitate its applications in pattern recognition. The work of this dissertation consists of three parts: (1) constructing nonlinear isometric mapping (i.e. diffeomorphism); (2) investigating the intrinsic geometry (i.e. intrinsic dimensionality, nonlinearity and geometrical model) of high dimensional dataset; and (3) attempting towards computational aesthetics. Specifically speaking, the following innovative works are achieved in this thesis:1. Proposed an E-ISOMAP algorithm, which aims at remedying the deficiency of lack of an explicit nonlinear isometric mapping in the original ISOMAP algorithm. Based on iterative majorization procedure, a version of ISOMAP algorithm with explicit nonlinear isometric mapping (E-ISOMAP) is presented and its supervised version (SE-ISOMAP) is also given. Owning to the existence of explicit isometric mapping, E-ISOMAP and SE-ISOMAP algorithms can be used for nonlinear feature extraction based on geodesic distance.2. Proposed a "two-step" approach for constructing the nonlinear isometric mapping of ISOMAP. Based on learning of parameterized geodesic distance function and constructing of distance-preserving mapping (i.e. triangulation), the nonlinear isometric mapping which is from high dimensional Euclidean space into low dimensional Euclidean space is constructed in an explicit way and a framework for feature extraction with ISOMAP is also formulated.3. Investigated the behavior of non-negative local linear reconstruction coefficients and discussed their possible applications for estimating the intrinsic dimensionality and discovering the subtle structure hiding in high dimensional datasets. Experimental results have shown that: (1) the number of the dominant non-negative local linear reconstruction coefficients indicates the intrinsic dimensionality of dataset provided the noisy level is low and the intrinsic dimension is small; (2) the non-negative local linear reconstruction coefficients can discover the subtle intrinsic structure hiding in dataset and hence can be used for conducting the pruning operation to improve the accuracy of geodesic distance based semi-supervised classification.4. Put forward a Principal Fiber Bundle (PFB) model assumption to formulate the intrinsic geometry of certain dataset, which consists of samples from multiple classes. Under PFB assumption, we presented a naive Bundle Manifold Learning (BML) algorithm, which utilizes the double neighborhood graphs, to discover the subtle structure hiding in dataset which lies on bundle manifold.5. Brought forward a novel research task, termed as Computational Aesthetics, which judges the handsomeness of Chinese Handwriting Character by computer automatically. Primary datasets HCL2000-CA towards to computational aesthetics are prepared and some exploratory experiments on HCL2000-CA dataset are given.