Facial Animation Under Arbitrary Illumination And Face Tracking

At first, this thesis provides a face tracking algorithm based on robust optical flow. The parametric motion models of human face are introduced firstly and then the face tracking system based on robust regression algorithm is developed. This face tracker enables the robust tracking of face with translation, rotation, zoom in/out, or partial occlusion.Then, this thesis presents a novel scheme about facial animation under arbitrary illumination and develops the corresponding tools. The facial animation scheme includes two parts: realistic facial rendering under changing illumination or with different expression, and adapted field morphing algorithm. In the first part, the reflection model of a particular face is captured firstly and structured as a collection of reflection functions. And then, with the help of LUXMASTER, the rendering tool, where a light map represents a lighting condition, a variety of faces with different expression are rendered under changing illumination. The second part is mainly about the morphing tool, FLUIDMAN, which can morph the rendered faces globally or locally. The combination of LUXMASTER and FLUIDMAN can produces facial animation under arbitrary illumination.Finally, this thesis introduces the model-based light map ( MBL ) technique and its basic idea and framework. In order to capture the facial reflection model efficiently, a 3D light stage with 1.5-meter radius is custom-built. A basic recovering algorithm based on conjunction gradient method is described and some results are provided. The basic idea of model-based light map is to explore the captured facial reflection model and recovering light map from a single picture. This technique can find applications in image composition and some facial animation system. For a picture with a face, MBL can recover the light map of the face and then a new face or any other synthetic object can be rendered into the scene smoothly in terms of illumination.