The Study Of The Manifold Learning Based Feature Extraction Methods And Their Applications

Recently, manifold learning has been attracting many attentions in the field of machine learning. Based on the assumption of local linearity and global non- linearity, manifold learning methods can explore and preserve the inherent structure of non-linear distributed data. So manifold learning based approaches are very efficient for data visualization. However, when encountering the tasks of classification, the original manifold learning methods generally show many shortcomings, such as small sample size (SSS), out-of-sample, sensitivity to noise and feeble discriminability. In order to overcome these problems, in this thesis, some new manifold learning based feature extraction methods were proposed. Particularly, a generalized Fisher linear feature extraction framework was constructed.The main work for this thesis can be summarized as follows:(1) Before manifold learning techniques are applied to extract features, generally it is very important to make a preprocessing to the data with noise. So a denoising approach based on Robust Principal Component Analysis (RPCA) was put forward in this thesis. Firstly, the optimal weight for each point can be computed by RPCA and Iterative Reweighed Least Square (IRLS); Secondly, Box method was adopted to distinguish the noise from the clear points based on these weights. Lastly, the original data after de-noising can be mapped to a low dimensional space using manifold learning based methods.(2) Locally Linear Embedding (LLE) is a classical manifold learning method, and it can project the original data into a low dimensional space by preserving the least reconstructed weights among the neighborhood points. Moreover, LLE has the property that the reconstructed error is invariant to rotation, translation and rescaling. In this thesis, the last two were introduced to improve the discriminant ability of the original data. Thus a new method, named as Locally Linear Discriminant Embedding (LLDE), was proposed to enhance the recognition ability of the original LLE, where a Modified Maximum Margin Criterion (MMMC) was adopted to automatically explore the optimal linear transformation for translation and rescaling. The experimental results on face data sets and gene expression data sets validated the efficiency of the proposed LLDE.(3) For non-linear distributed data, the task of classification can be viewed as classification-oriented multiple sub-manifolds learning problem. In order to overcome the problem, a constrained objective function model was constructed, where the constraint is to preserve the local structures while the objective function is to maximize the variance between sub-manifolds. The model has been named as Constrained Maximum Variance Mapping (CMVM), which seeks to find an optimal subspace where data in different sun-manifolds are located further and data in the same sub-manifold are clustered closer. It has been validated that CMVM is effective and efficient by the experiments on some face data and handwriting digital data.(4) Currently, most linear feature extraction methods, either the original linear ones or the linear approximations to the original manifold learning approaches, have a property in common that their objective functions can all be deduced to have the form of Fisher under different circumstances. Based on this property, a Generalized Fisher Framework (GFF) was put forward in this thesis, where both class label information and local structure information have been integrated together. When some constraints are satisfied and the parameters have been correctly set, these linear methods can all be taken as the special cases of GFF. The experimental results have also verified that our proposed algorithm is efficient and feasible.