3D Face Modeling And Standardization Based On RGB-D Data

Human face is one of the most important features of human, as well as an essential carrier of emotional expression.3D face modeling becomes a hot research area in computer vision and computer graphics due to its wide applications in face detection, face recognition, computer animation and other aspects of image processing.Although we can capture 3D models of faces in high resolutions directly by using 3D laser-scanners, these 3D scanners are bulky, costly, and highly demanding for acquisition conditions and computational resources, which greatly limits the practical applications of 3D face. The recently widespread consumer RGB-D cameras (such as Microsoft Kinect) are cheap and able to obtain the real-time RGB images and depth images though their spatial resolution is lower, which is expected to be an actual effective mode to put 3D face modeling into practical applications. This thesis aims at studying 3D face modeling, standardization and database construction based on consumer RGB-D cameras.In this thesis, the main structure and related working technologies of the Kinect cameras, which are used to acquire RGB-D data, is firstly introduced. Then the Kinect developing environment is established. After that, the methods of capturing RGB-D data are introduced, and the 3D face modeling process based on RGB-D data is designed and a system based on this method is developed. A RGB image based face correction algorithm which is Self-dependent Pose Correction is proposed for correcting the posture of those acquired 3D face model. And then, an experiment is designed to verify the proposed algorithm.In order to solve the 3D face alignment problem of the low-resolution RGB-D 3D faces captured by Kinect while building a RGB-D based 3D face database, a 2D resizable template-based alignment algorithm is proposed. Its idea is that the size of the 2D template is determined dynamically by the vertex number of the 3D face. Then, a high-resolution 3D face database is used to generate a 2D average template. Finally, the resample algorithm is used to automatically achieve the pixel-wise correspondence between 3D faces which is based on the template generated by the high-resolution 3D face database. Experimental results show that this method can not only construct the linear 3D face model rapidly and automatically, but also remain the main information of the original low-resolution 3D face.Finally, with the above techniques, a database for the face recognition and modeling in a free scene named SWJTU Multimodal Face Database is proposed. It contains face data acquired from 200 Chinese people with neutral expressions by using 4 different kinds of devices. The collected data includes visible light images, video sequences,3D face models (high resolution) and stereo video sequences which has three characters:1) Different modals. It contains two-dimensional image data and three-dimensional shape data; 2) Different resolutions and accuracy. It not only collects face images having different resolutions, but also collects face models with different accuracy; 3) A combination of dynamic and static face data. It includes both dynamic and static face data. This database provides a standard evaluation data resource for the study of 2D/3D face recognition,2D/3D face posture analysis,3D face reconstruction as well as facial feature point extraction.