High-dimensional-space Discriminant Analysis And Multi-manifold Learning

Discriminant analysis is an important research in the field of patternrecognition. The extraction of discriminative features is the key step of dataclassification. Recent years, manifold learning has become a hot topic inmachine learning and pattern recognition field. Its objective is to find the lowdimensional manifold embedded in the high dimensional space. Aiming at thehigh dimensionality, limited quantity, nonlinearity and unstructuredcharacteristic of samples, this work discussed the theory and method ofdimension reduction and discriminant analysis in depth. Moreover, weextended manifold learning to muli-manifold based. The main contributionsof the thesis are as follows:1. Semi-supervised marginal discriminant analysis (SMDA) and its fastalgorithm. SMDA aims at maintaining the intrinsic neighborhood relationsbetween the data points from the same class, while maximizing the marginbetween the neighboring data points with different class labels. Moreover, it isdesigned for semi-supervised learning which incorporates both labeled andunlabeled data. QR decomposition is then employed to find the optimaltransformation which makes the algorithm scalable and more efficient.Experiments on face recognition are presented to show the effectiveness ofthe method.2. Combing the idea of nonlinear kernel mapping, an extention of SMDAcalled kernel SMDA (KSMDA) was proposed. Samples are projected to highdimensional spaces and nonlinear features are extracted for recognition. Byintroducing the correlation metric, we further proposed a novel discriminantlearning algorithm in the correlation measure space, named correlation-based Semi-supervised Marginal Discriminant Analysis (CSMDA) which is shownto be generally better than Euclidean distance for classification purpose inmany real world applications.3. A Dynamic Graph Embedding (DGE) algorithm was proposed.Moreover, a semi-supervised discriminant analysis based on Grassmannmanifold was presented for image set matching. Specifically, DGEsimultaneously minimise the distance between images from the samesequence and maximise the distance between images from different sequenceto involve the dynamic information. By representing the image sets as pointson the Grassmann manifolds, through Grassmann kernel, discriminantanalysis could be easily extended to images sets. Different from othermethods on Grassmann manifold, the local structure of data is taking intoaccount.4. Inovatively proposed the multi-manifold assumption. Unlike mostprevious manifold-based data classification algorithms that assume all thedata points are on a single manifold, we expect that data from different classesmay reside on different manifolds of possibly different dimensionality. Thekey issues are discussed and a general framework for data classification onmultiple manifolds is presented. The intrinsic features are firstly learned foreach class separately, and a stochastic optimization algorithm is thenemployed to get the near optimal dimensionality of each manifold from theclassification viewpoint. Under this framework, data can be easily classifiedaccording to the minimum reconstruction error on manifolds.5. Manifold learning has been widely applied to facial expressionrecognition by modeling different expressions as a smooth manifoldembedded in a high dimensional space. In this paper, a generalized frameworkfor modeling and recognizing facial expressions on multiple manifolds ispresented. The steps of feature extraction, expression learning andclassification were discussed in detail. Extensive experiments on both theCohn-Kanade and Feedtum databases show the effectiveness of the proposedmultiple manifold based approach.