| This dissertation takes the feature extraction approach to dimensionality reduction by studying nonlinear manifolds for characterizing high-dimensional data. Among various nonlinear dimensionality reduction techniques, the principal curve, which is a nonlinear generalization of principal components, was found to be the most intuitive, interpretable, and theoretically elegant. However, the original principal curve formulation suffers from a number of problems: it is nonparametric, biased, inefficient, and not guaranteed to converge. To address some of these problems, I first develop two new principal curve algorithms—the Hastie-Stuetzle-Banfield-Raftery principal curve, and the probabilistic principal curve. The improved algorithms are then applied to feature extraction and classification of moderate-dimensional data with good results. Next, I develop the probabilistic principal surface (PPS), a general framework for computing principal surfaces of arbitrary dimensionality and topology. In addition, a generalized expectation maximization algorithm with guaranteed convergence is derived for the PPS. Unlike any of the existing principal surface approximation algorithms, the PPS is the only formulation that is self-consistent, parametric, unbiased, efficient, and guaranteed to converge. Empirical properties of the PPS are also studied and shown to be superior to the generative topographical mapping, which is a special case of the PPS.; With the groundwork laid for a flexible and powerful manifold formulation, three applications of the PPS are investigated. First, a spherical PPS is proposed for emulating the sparsity and periphery properties of high-dimensional data. It is found to be very effective for visualizing high-dimensional data; providing extra information over linear projection subspace methods such as principal component analysis. Second, a template-based classifier using spherical PPS as reference manifolds is developed and shown to compare favorably against traditional classifiers based on the k-nearest neighbor and Gaussian mixture models. Finally, a practical pattern recognition application of the spherical PPS for simultaneous classification and pose estimation of 3-D objects from 2-D silhouettes, is demonstrated. Results on simulated and real vehicle and aircraft images show the proposed approach to be highly accurate and effective. |
