Realistic 3D Face Modeling And Its Application

Realistic 3D face modeling and animation is the research focus in computer graphics and computer vision, and has many challenging open problems. It can be widely used in many areas, such as film, advertisement, character animation, computer game, teleconference, human computer interface, medicine, and safeguard etc. Recently, a new face modeling method, the Morphable Model is proposed. Comparing with the classic face modeling methods, such as the parametric model, the physical model and the deformable model, the Morphable Model has advantages in realism and automation, while there are many aspects of the model to be improved and explored, for examples, the model construction, the model matching, the lighting model and its application. Our work concentrates on the construction and application of the Morphable Model. The main study includes the following contents: (1) The construction of the Chinese 3D face database with large quantity. The 3D face database is the key step to the Morphable Model that demands many prototypic 3D faces as the base of the face space. The construction of the 3D face database includes prototypes acquiring, face data preprocessing and standardizing, and structure designing of the database. The difficult problem, the pixel-to-pixel alignments among the dense facial mesh of the prototypes must be analyzed and resolved. (2) The research on the Morphable Model and its improvement. The Morphable Model is implemented firstly. The implementation consists of the model construction and the model matching to the given facial image. The model construction includes the vectorization of the prototypes and the construction of the parametric facial models. The illumination, camera calibration and optimization with massive data are investigated in the model matching. On the other hand, the Morphable Model is considered to be improved in the matching with abnormal illumination and finding the faster matching algorithm. (3) The research on the application of the Morphable Model. We consider three applications of the Morphable Model: face animation, face recognition and facial feature extraction. In face animation, we want to find simple method to produce realistic facial animation with the Morphable Model. It is a good idea using the 3D face modeling method to solve the pose and light problems in face recognition. Applying the Morphable Model to facial feature extraction is inspired by its basic conception, the linear combination. As result of the research on the above areas, we have constructed a large scale 3D face database and the Morphable Model is constructed from the prototypic faces. By the Morphable Model we can reconstruct the 3D face from the given 2D facial image. Some difficult problems about the face modeling method are also resolved. The main contributions and innovations of our research are listed below: (1) Proposing a 3D face alignment method base on mesh resampling. Firstly, the proposed meshing resampling method gives a resolution to the difficult problem of the pixel-to-pixel alignments among the prototypic faces. It is important to the construction of the Morphable Model. Secondly, from the coarse to fine resampling strategy, a multi-resolution Morphable Model is constructed and is used in accelerating the model matching. Lastly, the mesh resampling method can be used in extracting the animation vectors between neutral and expressive faces. (2) Improving the original Morphable Model by the proposed multi-resolution and multi-light model. Considering the time-consuming model matching and the disadvantage in matching case with abnormal illumination, the Morphable Model is improved in two aspects. The multi-resolution face model is proposed to accelerate the model matching with more stability. And the Morphable Model is driven to fit the matching problem with abnormal illumination by the designed multi-light models. (3) Proposing a general face animation method based on the Morphable Model.