3D Face Modeling Based On A 2D Image

Human face is the most significant and essential carrier of emotional expression and people's daily communication. There are broad potential applications to model realistic 3D human face automatically using computer techniques, which is a challenging topic in the fields of computer graphics (CG), computer vision (CV) and artificial intelligence (AI). Human beings are born with a capacity of recovering 3D information from single image by human visual system (HVS). It is an essential problem in cognitive computing to develop new technologies for modeling 3D human face from a 2D image through simulating the cognitive procedure of HVS.Capturing shape and texture information using 3D laser-scanner is a straightforward method which is more accurate than others. Unfortunately, it can only be installed in a certain special situations due to the high cost of the equipments and its inflexibility. Therefore, current hotspots of face modeling focus on image based methods. However, high complexities of feature matching process and inefficiency are the main limitations of these methods. Modeling a 3D face from a few feature points on one image is a promising direction that could run quickly and automatically. Nevertheless, the primary challenge of this method is how to make full use of the small quantity of facial information to improve quality of the reconstructed 3D face.In this thesis, a human facial shape statistic model——extracted from a 3Dface database——is built as a prior knowledge to constrain the procedure of facemodeling. The main content of this thesis includes: establish a statistic model of human face shape by creating a standardized 3D face database; develop effective modeling algorithms and schemes for 3D face modeling; explore 3D face applications. And, the main contributions are listed as follows:1) By solving the correspondence between faces automatically, a 2D templatebased alignment algorithm is developed to create a standardized 3D facedatabase.It's a fundamental job to create a standardized 3D face database for building a face statistic model and for face modeling and animation. Inspired by the idea of mesh re-sampling, we propose a novel 2D template based alignment algorithm which could be implemented automatically and overcomes the main shortcomings of traditional methods, i.e., imprecise and operational complexity. Experiment results show that the standardized database created by our method has considerable good correspondence effects. It is easy to create a statistic model of facial shape, which lays a good foundation for 3D face modeling. (Chapter 2)2) Develop 3D face deformation algorithms based on features of a 2D image.A global deformation algorithm—Dynamic Component based Deformation Model (DCDM) is proposed, which could improve the precise and stability of deformation algorithm by selecting the most effective components in face modeling. Principal Component Analysis (PCA) based methods always use the prior components (eigenvectors) with the maximum eigenvalues to construct an eigenmatrix. This conventional strategy may import some irrelevant factors, or lose some useful ones, resulting in errors for the modeling process. We propose a dynamic component based deformation model that uses t-test to determine the correlativity for each component to the novel face at first, and then concatenate the most correlative ones to form an eigenmatrix. Experiment results show that the faces modeled by DCDM are more stable and more accurate. (Section 3.3)A local deformation algorithm based on prior knowledge—Sibson Weighted Local Feature Analysis (SWLFA)—is proposed, which could create smooth 3D face shape by allocating weights for each control point used by LFA. Since 3D faces created by global deformation method lack some personalities, we apply Sibson coordinate to local features analysis, a local deformation algorithm (SWLFA) is hence proposed based on prior knowledge of facial shape. SWLFA has strong local properties by eliminating the interactional impacts between control points. It could elaborately depict personality traits on human faces benefiting from making full use of the prior assumptions regarding facial characteristics. (Section 3.4)3) A Two-Step Face Modeling (TSFM) scheme is proposed to improve the fitting result in the direction of Z-axis. This is achieved by estimating features' depth from prior knowledge of human shape. We find that improvements are limited by just tuning the performance of deformation methods due to the lack of depth information of the facial features from only one image. By investigating three methods for feature's depth estimation, we indicate that the proposed Sparse Linear Model based Optimization in this thesis is more accurate and stable than the other two. Comparison tests show that, the estimated depth information can be used to improve the accuracy of human face reconstruction. Moreover, TSFM could improve the precision of facial shape reconstruction of both interpolation methods and statistical deformation methods. (Section 4.2)4) Based on a reconstructed 3D face, the problems of illumination, pose estimation and 3D face animation are further studied.The normal of each pixel on a 2D image face is estimated by establishing the correspondence between a 3D average face and the input facial image. Then, the spherical harmonic model is used for calculating and compensating illumination condition of the given image. Experiment results show that after illumination compensation, the recognition rate is significantly improved. (Section 5.2)Based on the point-to-point relationship between features on 3D face model and 2D image, we adopt a linear regression model to estimate the pose of the head on the input image. The comparison results show that the revolving angles estimated by our algorithm are more accurate than existing methods on both multi-axis and single axis. (Section 5.3)A three-layer control model is adopted to generate new expressions based on 3 levels—features, organs and expression. Base upon this, a 3D face animation system is developed. By using the MPGE4 standard, our system is highly automatic and general-purpose. Experiment results show that realistic 3D expressions animation could be generated by this system. (Chapter 6)...