Research On Statistics-Based Realistic Face Image Synthesis

Realistic face image synthesis appears as an important technique in the field of human computer interaction. It is also an active research topic both in computer vision and computer graphics community. The potential application of this technique includes low bit-rate video transmission, computer aided instruction, game design, virtual reality and so on.Traditional approaches use wire-frame model or surface model to build 3D head structure, and use physiological model or parametric model to make facial animation. However, there are many problems to be solved with it on the acquisition of 3D data, the accuracy of model representation, the complexity and the robustness of algorithms. Most recently, the example-based approaches are also actively adopted in face synthesis. This new strategy, utilizing example images directly, without any 3D reconstruction, can often achieve more realistic effects than traditional methods.This thesis focuses on example-based face synthesis, discusses how to utilize training examples and statistic methods to synthesize face image in a wide range of view, under different lighting condition, and with various emotional expressions. We summarize the relevant literatures first, and then propose our novel ideas and show the exciting results by interesting experiments. Our work demonstrates the advantage of statistical learning theory in solving problem of face image synthesis. It also points out a new direction of development in the field of face modeling and animation.The main contributions of this work are listed as follows.First, a multi-view face synthesis method based on factorization model is proposed. Here "human identity" and "head pose" are regarded as two influence factors, and their interaction is trained with a face database. With the special ability provided by factorization model, a test face can be translated into old views contained in training faces, and training faces can also be translated into new view of the test face. The original bilinear factorization model is also extended to nonlinear case so that global optimum solution can be found in solving "translation" task. Thus, with a pre-processing and a post-processing procedure, an arbitrary new face is able to be translated into other views. The proposed method can be applied to areas such as multi-view face database building and face recognition across a wide range of view.Second, a method of dynamic facial texture generation based on shape appearance dependence mapping is proposed. It has been proved that there is a high correlation between shape and texture of facial features. Based on this observation, the dynamic facial texture can be generated according to shape variation and this strategy is called dependence mapping. We implement the dependence mapping on mouth region and show that a realistic mouth animation can be generated according to several shape parameters. We also test performance of the mapping by using a video clip of facial expression. The experiment shows that the expressive details are successfully recovered from the movement of facial feature points. The proposed technique can be integrated to a talking head system to generate realistic animation, or applied to a model-based coding system to produce more efficient bit-rates.Third, a simple methodology for mimicking realistic face by manipulating emotional status is proposed. A mapping from emotional status to facial expression is trained with a face database. The mapping is called "emotional function" which can be used to generate expressive face for new person by utilizing just his/her neutral image. In fact the emotional function describes the way of expression variation according to inner emotion. Because of this, the function can be trained with different training set to reflect different affective style. While building the statistical model for face image, the model is trained in relative way rather than in original way. This training strategy makes expressive details extracted in person independent manner. As the experi...