3D Facial Data Acquisition Based On 2D Images

Based on the image processing and binocular stereovision theory,in this paper we have present a system to generate 3D facial data form two photos of face and implement it with VC++ 6.0 on the platform of Windows. The system first adopts two cameras to shoot the face of the same person form different angles re-spectively,then finds a few corresponding point matches between two the pictures, which are later propagated to be quasi-dense , integrating the camera's geometry parameters,the 3D facial data is calculated. Applying the clue of the whole system frame,the basic idea and arithmetic of developing the module block are presented, including the calibration of camera, the stereo matching, and the acquisition of 3D point cloud.Moreover,parts of the experimental results are presented.The first step is the calibration of the camera. The relation between the position of a 3D point and its camera image is determined by the camera imaging model. The parameters of the model is called the camera parameters,and the resolution of these parameters is called the calibration of the camera.Our system uses Olympus C-770 digital camera, and resolve the camera's interior and external parameters using Zhang Zhengyou camera calibration technology [1].The main idea of the arithmetic is as follows: Print a pattern and attach it onto a planer surface (the pattern is as figure 4); Shoot it with two cameras form two different angles; Detect the corners (e.g, the corner of the black squares );Resolve the interior and external parameters according the physical and image coordinates of the same corners; Resolve the distortion parameters; Optimization.Acquiring the physical and image coordinates as accurate as possible and match- ing them is the most difficult part in Zhang's arithmetic.As to the pattern, we construct it using Photoshop as follows: the black square is 1cm×1cm, the space between the squares is 1cm. Print it and attach it onto a glass board.Considering the center of the square in the 7th row and 7th column as the origin of the world coordinate system, the positive direction of x axis rightward, and the positive direction of y axis upward, the physical coordinates can be easily determined.With respect to the acquisition of the image coordinate ,we mainly using Harris corner detection arithmetic and line fitting technology.Fisrtly,we use Harris' arithmetic to detect the corners,secondly transfer the image to be binary image with threshold processing technique.then eliminate the imprecise corners and rearrange the corners left,finally fit a line according to every row and column of corners. The coordinates of the points of intersection of these lines are the subpixel image coordinates of corners.In the module of stereo matching, we proposed a new structural color light projecting technique based on projector. The arithmetic is easy and efficient. The main idea of the arithmetic is as follows:Project the color pattern onto the face of the plaster model;Take two pictures from different angles; Preprocess the pictures; Starting at he origin node which is purple, find the other color nodes as many as possible in the two pictures respectively, then store them in two lists, whose head pointer are OriginNode *head1 and OriginNode *head2 ; Traverse these two lists simultaneously,the nodes whose color and coordinates are identical are regarded as the corresponding pixels and store their image coordinates ; Eliminate the bad matches and optimize the left according to epipolar geometry .The pictures need preprocessing before find matching , because there is a lot of noise during the acquisition of the pictures. Our preprocessing mainly include eliminating the effect of illumination and background. Concerning the illumina-tion,we pick the " best picture" ,namely the picture form which we extract the most accurate color node,transform the grey-scale of the other images in order that the mean and standard deviation are identical to these of the "best picture" .As to the background,set the background to be black so that the mean and standard deviation can reflect the facial grey-scale.The arithmetic also use epipolar geometry to eliminate the bad matches and optimize the matches left. The strategy is as follows: given a threshold T,which is decided by experiment,the distance between a point and its match point's corresponding epipolar line is Dist1,the opposite direction is Dist2 , let Dist = Dist1+Dist2, if Dist is lager than T, the point match is eliminated, otherwise, optimize it by looking for the "best" point match in their neighbourhood whose Dist is the shortest.The primary point matches are sparse and is far away from the need of face recognition and other applications.So the system later propagate them to be qusi-dense.The improved propagation arithmetic is adopted, [2], [3]. The main idea is that: pick the point match whose ZNCC is largest , and look for the potential matches (u, u') in its immediate neighbourhood N(x, x') following some particular "matching rules" ,as figure 13.Given that the facial texture is not obvious, we introduce image gradient together with ZNCC. Most of the threshold of the original arithmetic is reset by experiment results. Moreover, epipolar geometry restriction is also applied.Finally , integrating the camera parameter and the image coordinates of the matches, 4 liner equations involving 3 unknown variables can be deduced . The world coordinates of the point cloud can be acquired by solving the equations using least square method.The system we proposed here process the pictures of several experiments, and the results are good. We can apply it in the research of 3D face recognition and 3D camera.