Research On Manifold Learning Theories, Methods And Applications

Applications in many domains always involve in data with high dimensionality,which covers up the intrinsic laws of data and leads to"curse of dimensionality"problem.Therefore, dimensionality reduction, which aims to reduce the dimensionality of data andrepresent it as low-dimensional coordinates, is appealing for extracting the useful andinteresting knowledge hidden in the data. Generally speaking, dimensionality reductioncan be divided into two categories: linear methods and nonlinear methods. Due to thelimitations of linear technologies in real applications, nonlinear methods obtain more andmore attentions from related communities. As a nonlinear technology, manifold learning,whichisbasedonmanifoldassumptionandhastheoreticalsupportfromhumancognition,has appeared in researchers'sights and become a hot research spot.Under this background, we has conducted some deep theoretical and applied inves-tigations in manifold learning to overcome the limitations of existing methods. The mainwork and contributions of this thesis are as follows:1. Proposing an adaptive neighborhood selection method to overcome the sensitive-nessofexistingmanifoldlearningalgorithmtotheparameterofneighborhoodsize. Basedon local smoothness of manifold, most of existing methods recover the manifold struc-ture via local linear fitting. Hence, they all involve in constructing neighborhood and aresensitive to the neighborhood size. In view of the purpose of constructing neighborhood,which is to linearly fit the constructed neighborhoods, an adaptive neighborhood selec-tion method is proposed. Under the premise that the constructed local neighborhoods areguaranteedtobeoneswithlinearstructure,theproposedmethodcanadaptivelydeterminethe neighborhood size according to geometric structure of manifold, then further reducelocal fitting error and promote the performance of manifold learning algorithms.2. Presenting a robust manifold learning algorithm. In real applications, the sampleddata are always corrupted by noise, which will destroy the local smoothness of manifoldand cause the sampled data points deviate from the underlying manifold. Hence, per-formance of existing methods, which are all based on strict manifold assumption, willbe significantly influenced by noise. In the context of Local tangent space alignmentalgorithm, we firstly carefully analyze the mechanism how the noise influences the per-formance. ThenweusetherobustPCAinsteadofSVDtocomputethelocaltangentspace coordinates. Finally we improve the LTSA from other two aspects to make it robust tonoise. The proposed method solves the noise manifold learning problem to some extent.3. In context of image set with missing pixels, a novel manifold algorithm to learnits manifold structure is proposed. To our best knowledge, there is no method that canlearn manifold structure from data with missing value. As a try, the thesis investigateshow to learn the manifold structure of image set with missing pixels. Based on the keyobservation that there is strong redundancy between pixels in image set with manifoldstructure, we propose a algorithm called EM-PCAM which can perform PCA on imageset with missing pixels by only using the known pixels. The EM-PCAM are seamlesslyintegrated into the LTSA algorithm such that it can learn the manifold structure of imageset with missing pixels.4. Combiningthetaskofclustering,twononlineardimensionalityreductionmethodscalled CPE and NDECSR are proposed. Classical manifold learning algorithms, whichare unsupervised learning methods, solely aim to recover the manifold structure. How-ever, in real applications one may be interested in classification and clustering rather thanthe geometric structure. Therefore, the thesis investigates the problem of combining themanifold learning and clustering, and proposes two nonlinear dimensionality reductionmethods. CPE obtains the low dimensional coordinates by trying to preserve the robustpath-based similarity whereas NDECSR obtains low dimensional coordinates and clus-tering results simultaneously by introducing spectral regularization.5. Taking the manifold structure of face image set into account, we propose a novelfeature extraction method based on maximum margin criterion and image matrix bidirec-tionalprojection. WeutilizethemanifoldinformationbyintroducingtheLaplacianmatrixwhich can characterize the manifold structure. In addition, to keep the structure informa-tionofimage, wedirectlyrepresenttheimageasmatrixandadoptbidirectionalprojectionto implement feature extraction. Different from Fisher criterion, the proposed method usemaximum margin criterion, which can guarantee a convergent iterative solving procedureand solve the problem of non-convergent solution of existing methods.