Image-based 3D Face Modeling

The development of computer hardware and software has made the 3D model become more and more popular in our life,work,and entertainment.How to acquire realistic 3D model is a major research topic in either computer vision or computer graphics.The image-based modeling is one of the major 3D acquisition measures. This thesis concentrates on the image-based 3D modeling of faces.Although face is just one kind of objects being modeled,the 3D modeling of faces has its unique characteristics compared to other objects.On one hand,3D face models play important roles in subjects such as face recognition and face animation,thus are highly correlated to these subjects.As a result,we should incorporate the successful research results of these subjects into the 3D face modeling.On the other hand,human face is an object which shares a lot of commonness across different individuals. This commonness is very helpful in extracting prior information or knowledge about faces to facilitate the 3D modeling procedure.This means that 3D face modeling involves methodologies borrowed from pattern recognition,statistical learning,not just the computer vision.Based on a thorough review of the state of art,we perform our research work on the following aspects.Regarding the face image preprocessing techniques,we propose a novel projection algorithm called Locally Selective Projection,and apply this algorithm to eye location. Compared with existing eye location methods,our method based locally selective projection shows higher performance against the changes in illumination and pose,and exhibits more precise eye detection results.We propose a fast 3D face modeling method based on the combination of shape-from -shading and radial-basis-function interpolation.This method utilizes the accurate 3D information provided by shape-from-shading on several facial feature points,and uses the 3D information of these feature points to drive a modification process from a general 3D face model to a specific one,based on the radial-basis-function interpola-tion. The proposed method can construct realistic 3D face model with low computa-tional expense. We also propose a novel shape-from-shading method which uses the statistical learning to learn a mapping from texture to depth images,and further uses the mapping to estimate depth from texture for 3D face modeling.We incorporate the face alignment techniques into our learning process to dramatically promote the depth estimation accuracy.We also introduce the kernel based learning to deal with the high dimension learning problem.Regarding the problem that the image-based 3D face modeling can be greatly influenced by illumination changes,we propose a solution of 3D face modeling from near infrared images.The near infrared images,due to its unique capture hardware,are immune from illumination changes,thus can be a qualified source for depth estimation. We build a database containing 3D faces and corresponding near infrared images,and propose an algorithm for aligning these two kinds of data.We then use statistical learning to learn the relation between near infrared images and depth images,and further use this relation for depth estimation and 3D face reconstruction.Stereo is an important 3D reconstruction method.Because the human faces are lowly textured,the conventional stereo methods based on texture correlation can not give satisfying 3D face reconstruction results.We propose a stereo matching method based on virtual stereo correspondences.The virtual face images with known correspondences are first synthesized from a reference 3D face.Then the known correspondences are extended to the incoming stereo face images,using face alignment and image warping.The 3D face point cloud can thus be reconstructed from stereo images reliably.Although the 3D point cloud provided by stereo is accurate,it is not smooth enough for the realistic rendering of 3D face.We present a novel and efficient method to obtain dense 3D face from stereo images by combining stereo vision and morphable model.Using morphable model as the reference face for the calculation of stereo matching,we can obtain the correspondences between morphable model and stereo point cloud.Then the stereo point cloud is further registered with the morphable model to obtain a refined face surface.The reconstructed face surface shows high fidelity and smoothness.In conclusion,in this thesis,we perform a thorough research on the topic of image based 3D face modeling.We hope that our work can be helpful for the relating researches.