Font Size: a A A

Finite Mixtures Of Models, Nonlinear Two-Dimensional Principal Component Analysis And Their Applications To Pattern Classification

Posted on:2006-02-10Degree:DoctorType:Dissertation
Country:ChinaCandidate:H X WangFull Text:PDF
GTID:1118360155961199Subject:Computer application technology
Abstract/Summary:PDF Full Text Request
Based on the theory of statistics, this dissertation investigates two aspects of unsupervised method: (a) the systematical study of some topics that arise in finite mixtures of models, and (b) the researches on nonlinear extensions to two-dimensional principal component analysis (2DPCA), during which we take face recognition into account. In statistical pattern recognition, the finite mixture model is a formal (i.e., model-based) approach to unsupervised classification, and the central issue of 2DPCA-based method is to explore feature extraction in an unsupervised way. The characteristic of unsupervised technique is to employ unlabelled samples to learn or extract features. The main research works and contributions of this dissertation are outlined as follows.Firstly, we consider mixtures of multivariate t distributions that belong to the family of heavy-tailed distributions. We often focus on mixtures of multivariate Gaussian distributions for clustering or fitting a set of multivariate data. But for many applied problems, the tails of Gaussian distribution are light than required. Also, the estimates of the component means and covariance matrices could be badly affected by observations that are atypical of the components in the Gaussian mixture model being fitted. Modelling mixtures of multivariate t distributions is usually adopted as a standard and robust alternative to Gaussian mixtures. Further, the multivariate data set often involves missing values inevitably. We present a framework for fitting mixtures of multivariate t distributions when data are missing at random on the basis of maximum likelihood (ML) estimation. We resort to expectation maximization (EM) algorithm for both the estimation of model parameters and the coping with missing values. The iterative algorithm obtained could be applied to an extensive range of unsupervised clustering as well as supervised discrimination.Secondly, we carry out researches on the estimation of the number of components g in a mixture model. It is an important issue with the fitting of finite mixture model, and some classical fitting methods (for example, ML, Bayesian approach) take effect only when g is specified beforehand. However, on many occasions, the number of components could not be made available, and so has to be inferred from the data obtained, along with the parameters in component densities. This is the question of model selection, and is also the main difficulty with EM algorithm for fitting mixture model, sinceEM algorithm itself could not estimate g, and, on the contrary, the algorithm needs g to be determined before the remaining parameters could be estimated. To solve the problem, an unsupervised algorithm for learning a finite mixture model, which is called stepwise split-and-merge EM (SSMEM) algorithm, is proposed. The adjective "unsupervised" is justified by three properties of the algorithm: (a) it uses unlabelled samples directly, (b) it is capable of selecting g automatically, and (c) it does not required careful initialization. The SSMEM algorithm alternately splits and merges components, estimating g and the other parameters in the components simultaneously. Also, two novel criteria are given to efficiently select the components for splitting or merging. Experimental results demonstrate the effectiveness of the proposed algorithm.Thirdly, we devote ourselves to the researches on probabilistic two-dimensional principal component analysis and their mixtures, incorporating the application to face recognition. Principal component analysis (or eigenfaces) has demonstrated its success in face recognition as a subspace method, and by now has become a de facto standard and a common performance benchmark in the field of face recognition. However, almost all of the methods based on eigenfaces operate in high-dimensional image space, which make it not easy to find the face space. The recently proposed two-dimensional principal component analysis (2DPCA) is based on original image matrices directly, and thus obviates the restriction of transformation from image matrices to vectors, which is required by the eigenfaces-based approaches. 2DPCA has proved to be an efficient technique for face recognition. By supposing a parametric Gaussian distribution over a new image space spanned by the row vectors of image matrices, and a spherical Gaussian noise model for the images, we endow the 2DPCA with a probabilistic framework called probabilistic 2DPCA (P2DPCA), and show that 2DPCA is a particular limiting case of P2DPCA. For complex database containing much variation of lighting conditions and poses, 2DPCA is possibly affected by them, especially the projection directions corresponding to small eigenvalues, while P2DPCA could suitably separate signal from noise. Further, P2DPCA is extended to a mixture of such models, which provides an approach to extracting nonlinear features. The parameters in proposed models could be iteratively calculated via EM algorithm on the basis of ML estimation. Experimental results on complex databases indicate the superiority of the proposed models.Finally, we shift our research of interests to kernel 2DPCA. A number of recently research efforts have shown that face images possibly reside on a nonlinear submanifold. However, 2DPCA is still a linear projection method, and the projection directions are based on the second order statistics of the image pixels, i.e., image sample covariance matrix. It does not address higher order statistical dependencies in an image, which include nonlinear relations among the pixel intensity values, such as the relationships among three or more pixels. For face recognition, much of the important information may be contained in these relationships. On account of the failure of 2DPCA in discovering the nonlinear structure hidden in the image space, we propose a kernel based 2DPCA (K2DPCA) by exploiting the kernel-trick, and so it extends 2DPCA to nonlinear space. Through the use of the polynomial kernel, higher order correlations could be captured among pixels of facial images, since it amounts to identifying the principal components within the product space of the rows of image matrices. With kernel-trick embedded in 2DPCA, K2DPCA is shown to be effective by experimental results.
Keywords/Search Tags:Unsupervised learning, Feature extraction, Finite mixtures of models, EM algorithm, Multivariate t distribution, Number of components, SSMEM algorithm, Face recognition, Eigenfaces, 2DPCA, P2DPCA, MP2DPCA, K2DPCA
PDF Full Text Request
Related items