Three-dimensional facial expression modeling and analysis with topographic information

Research on face tracking, modeling and animation techniques has been intensified due to its wide range of applications. There is a long history of interest in the problem of recognizing human emotion from facial expressions. My work focuses on the detection and description of facial expression features. First we introduce the new face tracking and modeling algorithm, which effectively extracted certain feature points, head pose orientation, and eye close-open status from input videos. Based on the detected facial features, a 3D face model is animated by a dynamic inference algorithm which transforms facial motion parameters to facial animation parameters. A factorization and synthesis based approach is introduced, together with our proposed topographic feature based modeling approach. As a result, given a face image at a front view, a realistic facial model can be generated using the extended topographic analysis and model instantiation approach. Secondly we present a novel 3D-based facial expression analysis algorithm to overcome the disadvantages of the traditional 2D-based analysis, which is incapable of handling large pose variations. 3D modeling techniques haven't been used on 3D facial expression recognition due to the lack of a publicly available 3D facial expression database. Thus we developed a 3D static facial expression database, including both prototypical 3D facial expression shapes and 2D facial textures of 2,500 static models from 100 subjects. This is the first attempt to create a 3D facial expression database for the research community. The last part of the thesis presents our study on the usefulness of such data for recognizing and understanding facial expressions. We investigated the importance and usefulness of 3D facial geometric shapes to represent and recognize facial expressions using 3D static facial expression range data. We extract primitive 3D facial expression features, and apply the feature distribution to classify the prototypic facial expressions. Experiments were conducted on person-independent facial expression recognition to validate the algorithm. The experimental result demonstrates the advantage of our 3D geometric based approach over 2D texture based approaches in conditions of head poses changes and illumination variations as well as a significant improvement on recognition accuracy.