3D Human Face Realistic And Dynamic Data Acquisition And Application

3D measurement technology is widely applied in many areas such as industry, aerospace, military, medicine, art and etc. Most applications of 3D measurement technology aim to get accurate 3D points of an object (such as Coordinate Measuring Machining) or reconstruct 3D model (such as Reverse Engineering) of it. Lots of research work is done for the two purposes and got exciting payback on accuracy, speed and etc. However, to some issues such as 3D face data acquisition for laser carving and 3D face dynamic data acquisition from videos, traditional 3D measurement cannot meet the requirements directly. Firstly, it cannot supply 3D releastic data with effects of light and shadow. Secondely, it cannot reconstruct a target automaticly along a video.The research in this dissertation right focuses on the 3D face releasic and dynamic data acquisition and application.For 3D releasitc data part, research can be described as how to use rapid, efficient, portrait-suitable measurement method, instrument and corresponding data processing method to get accuract enough and realistic 3D portrait data; how to use efficient, less time-consuming method to carve the 3D portrait data inside a glass cube for exhibination with laser. The word "realistic" here does not mean how vivid and real the reconstructed 3D face model looks by use of color, texture or such elements, while means considering the profile and effect of light and shadow how real and natural the 3D portrait which is comprised of points in a glass cube looks; For dynamic data part, research can be described as dynamic (in videos) 3D measurement of feature points (markers) on human face, in other word, how to find and keep correspondence of feature points in videos from multiple cameras. These points are closely related to human expression change, so collecting dynamic movement of these 3D points is equal to biluding a 3D face expression database for 3D affect recognition researchers. During the research on above subjects, the following results are achieved:(1) A new grating projection 3D measurement method based on calibration and structured light.A new grating projection 3D measurement method which carefully considers the featured requirements of face measuring is proposed. It is based on camera (single) and projector (light planes) calibration, incorporation of computer vision technology and structured light. To a 2D point in a grating strip, a 3D ray can be constructed based on the camera's intrinsic and extrinsic parameters; Also a 3D plane can be decided by checking out the point is in which strip as well as in which light plane; Then the intersection of the ray and the plane is just the expected 3D point. The instrument is not only capable of doing the 3D measurement, but also designed to meet the requirement of getting 3D realistic portrait points. It controls and coordinates the camera, projector and the lighting device to take two consequent photos on a subject with different settings in a very short time: the first one with grating projected to the subject's face for collecting 3D information; the second one with normal lighting, without grating projected, for collecting grayscale information.(2) A new method to acquire 3D realistic portrait data.The new 3D measurement method mentioned above can get human face 3D data rapidly, but those 3D points are distributed in lines. If the orginal 3D points are carved directly into a glass cube with laser, the portrait does not look real and natural. However, we can use the first photo to collect original 3D points and reconstruct the corresponding 3D face model; then project all white pixels of the dithered second photo to the face model to get qualified 3D points for laser carving. The new-generated 3D points will make the portrait not only three-dimentional but also releastic with effect of light and shadow.(3) Dynamic-connecting method for fast reconstruction of 3D points.To get releastic 3D face data, after the original 3D data obtained by grating projection 3D measurement method, face surface model need to be reconstructed. In a manner, the reconstructed surface is the bridge connecting orinigal and releastic 3D face data. For 3D points distributed in regular pattern, general algrithoms for scattered point reconstruction which is often more complicated, time consuming and sometimes needs external intervention is not the best choice here. Considering the priori knowledge of the distribution, a more simple, fast and automatic method called dynamic-connecting method can be used for surface reconstruction.(4) A new method for optimization of 3D laser-carving path based on balanced focusing energy.To decrease the time spent in carving the final realistic 3D points in a glass cube, path optimization adjusts the carving order of each 3D point. Usually, for the carving quality, all the 3D points should be classified into layers first and then carved layer by layer. Path of point carving in each layer is optimized locally. If without loss of quality, the path optimization could be done across some neighbor layers, more time cost would be cut down because of the improvement of total opitimization. In order to increase the number of points and make more layers involved in once local optimization, a method is proposed to estimate if points in different neighbor layers are covering each other and quantize the covering impact.(5) A new prototype for dynamic data collection of 3D face feature points database.In this prototype, five synchronized cameras distributed in a half circle (radius: 1 meter) construct four stereo vision system groups to cover the whole face area for 3D measurement. Stick different colored markers to the positions where the feature points (points are closely related to human expression change) are on a subject's face. Each group is in charge of markers dynamic 3D reconstruction (in videos) in corresponding fixed area and all the final data will be transformed into a same world coordinate system. For a marker, when the first frame is coming, give it a unique ID manually first, then track it in the following frames from two related cameras. The correspondence of this marker in different videos can be kept all the way for reconstruction in each frame. Considering the good environment, Camshift method is used for tracking and Kalman filter is used for result data filtering. To deal with loss of tracking, each marker's tracking position (after filtered) in two relate cameras are verified by the epipolar line constraint to decide if need to trigger a warning.