Multi-view face processing in video imagery

Face processing can be applied in many fields, such as surveillance, human computer interaction, and entertainment. The goal of this research is to develop novel algorithms for multi-view face processing in video imagery. Specifically, this research focuses on detecting and tracking multi-view faces, as well as on performance modeling and prediction of face recognition systems.; This work presents a recursive nonparametric discriminant analysis (RNDA) method to extract powerful features for multi-view face and eye detection. Based on extracted RNDA features, probabilistic classifiers are constructed and combined together to form a strong classifier using Adaboost. Compared to commonly used Haar wavelet features, RNDA features show better accuracy in detecting complex objects, such as profile faces and eyes. Experiments demonstrate that face recognitions using our automatically localized eyes provide accuracy comparable to those using manually marked eyes.; To address the drifting problems associated with the traditional object tracking under significant changes in both objects and environmental conditions, this dissertation presents a collaborative tracking method, which probabilistically combines measurements from specific face models with measurements from a generic face model in a dynamic Bayesian network for robust multi-view face tracking. In addition, the probabilistic tracking results are used to incrementally adapt the specific face models to individuals. Experimental results demonstrate that the collaborative tracking method can handle large face pose changes, and can efficiently build specific face models online.; To predict the success or failure of face recognition systems on a detected or tracked face, generic methods are presented to model and predict the face recognition performance based on similarity scores. A performance metric extracted from perfect recognition similarity scores (PRSS) allows modeling the face recognition performance without empirical testing. Features are extracted from similarity scores to predict recognition results of individual or sets of query data online. The presented methods can select algorithm parameters offline to achieve near optimal accuracy under different environments, predict recognition performance online, and adjust face alignment online for better recognition.; This research develops novel solutions to enhance several important components of a face processing system. The combination of all the innovations allows building a robust and fully automatic human face processing system.