Statistical modeling of video event mining

Video events contain rich semantic information. Using computational approaches to analyze video events is very important for many applications due to the desire to interpret digital data in a way that is consistent with human knowledge. This thesis investigates object-based video event analysis based on a statistical framework.; Within the proposed architecture for object-based video event understanding, object detection is addressed by model-based approaches with the integration of prior color/shape knowledge and recognition feedback. Object classification is investigated as shape-based image retrieval. The relevance feedback is used to adaptively derive basis vectors to capture a user's perceptual preferences. The major focus of this thesis is concerned with statistical modeling of facial event recognition. Two hidden Markov model (HMM) based approaches are presented. The first approach tracks the deformation of facial components in image sequences via active shape models (ASMs) and extracts geometric-based features for facial gestures. The interaction between upper and lower facial components is explicitly modeled via coupled HMMs (CHMMs) by introducing coupled dependencies between hidden variables. The second approach automatically locates face regions in each image frame via eigenanalysis, extracts multi-band appearance features based on Gabor filtering, and models the spatio-temporal stochastic structure of facial image sequences using hierarchical HMMs (HHMMs).; The major contributions of this thesis include: (1) a fully automatic person-independent facial expression recognition prototype system; (2) modeling of the spatio-temporal structure of facial image sequences within a hierarchical framework; (3) derived generalized inference and learning algorithms of HHMMs for observation sequences with known multiscale structures; (4) improved performance of ASMs for facial component tracking by using dynamic programming based search with contextual constraints; (5) explicit modeling of the interaction between upper and lower facial components via CHMMs for facial gesture recognition.; The proposed architecture provides a feasible method for object-based video event analysis. In particular, the proposed statistical models provide more accurate modeling schemes than conventional HMMs for characterizing facial image sequences and experimentally demonstrate their advantages for facial gesture/expression recognition. Although only demonstrated on the problem of facial event recognition, the proposed approach can be extended to general video event understanding.