Dimensionality Reduction Algorithms Based On Manifold Learning And Their Applications To Face Recognition

In recent years, with the development of science and technology, it is more easy and convenient for us to obtain various data. However, the data sets collected from the real application problems are always with the characters of high-dimensional and nonlinear. On the one hand, these characters cause the"curse of dimensionality"phenomenon; on the other hand, it is hard for us to understand and discover the intrinsic structure of the data set. As a result, using dimensionality reduction techniques for data processing becomes extremely important. Although the traditional dimensionality reduction methods (such as principal component analysis, independent component analysis and linear discriminant analysis) can effectively deal with the data set with linear structure and Gaussian distribution, they cannot discover the intrinsic nonlinear information hidden in the high-dimensional data set. The dimensionality reduction methods based on manifold learning assume that the high-dimensional observations can be modeled as data points reside on a low-dimensional nonlinear manifold, which is embedded into a high-dimensional Euclidean space. Therefore, the manifold learning methods can effectively discover and preserve the curve structure of the input data. At present, manifold learning has become a hot research topic in the fields of data mining, pattern recognition, machine learning, etc.In this thesis, we analyze the manifold learning and propose three novel manifold learning methods for dimensionality reduction and feature extraction, and apply them to the face recognition problem. The efficiency of the proposed algorithms is demonstrated by extensive experiments and comparison with other algorithms. The main work and contributions of this thesis are summarized as follows:1.A survey of the existing dimensionality reduction methods is given, and a brief introduction on the definition and applications of manifold learning is also made. Through the analysis of the face recognition problem, the rationality and feasibility of utilizing the manifold learning methods for face recognition are justified.2.A supervised manifold learning algorithm based on patches alignment is proposed. In this algorithm, we first chooses a set of overlapping patches which cover all data points using a minimum set cover algorithm with geodesic distance constraint. Then, principal component analysis (PCA) is applied on each patch to obtain the data's local representations. Finally, patches alignment technique combined with modified maximum margin criterion (MMC) is used to yield the discriminant global embedding. The proposed method takes both label information and structure of manifold into account, thus it can maximize the dissimilarities between different classes and preserve data's intrinsic structures simultaneously. Experimental results show that the proposed algorithm achieves better recognition rates than some existing methods for face recognition.3.An adaptively weighted sub-pattern locality preserving projection (Aw-SpLPP) algorithm is proposed. Unlike the traditional LPP algorithm which operates directly on the whole input patterns and obtains a global features that best detects the essential nonlinear manifold structure, the proposed Aw-SpLPP method operates on sub-patterns partitioned from an original whole patterns and separately extracts corresponding local sub-features from them. Furthermore, the contribution of each sub-pattern can be adaptively computed by Aw-SpLPP in order to enhance the robustness of the proposed method. Through applying the Aw-SpLPP to face recognition, it can be seen that the proposed method can not only reduce the computation complexity of the traditional LPP, but also improve the recognition performances.4.A novel local matching method called structure-preserved projections (SPP) is proposed. Unlike most existing local matching methods which neglect the interactions of different sub-pattern sets during feature extraction, i.e., they assume different sub-pattern sets are independent; SPP takes the holistic context of the original whole patterns into account and can preserve the configural structure of each input pattern in subspace. Moreover, the intrinsic manifold structure of the sub-pattern sets can also be preserved in our method. Like the two aforementioned algorithms, we also apply the SPP to face recognition problem. The efficiency of the proposed algorithm is demonstrated by extensive experiments on three standard face databases (Yale, Extended YaleB and PIE). Experimental results show that SPP outperforms other holistic and local matching methods.