Face Reconstruction And Recognition Based On Binocular Stereo

Face is an indispensable object in daily life. As an important branch of Biometric Identification Technology, face recognition has a wide application in identity authentication, video surveillance, security, anti-terrorism and human-computer interaction. Moreover, the relevant theories, methods and algorithms derived from face recognition technique have a significant guiding effect to pattern recognition, and lay a solid foundation for future breakthroughs in some key technologies and research topics. Although face recognition has been paid extensive attention by researchers in recent decades, and has been made great progress indeed, it is undeniable that face recognition is still facing huge challenges. One of the main challenges is that, differences between captured face images of the same person under uncontrollable illumination conditions and variant face poses, are much larger than differences between captured images of different face under the same illumination condition and face pose.With the constant improvement of computer performance and the development of 3D data acquisition technology, 3D face data has been integrated into face recognition system, and provides a basis for system to extract the essence of human shape so as to realize robust face recognition under different application environment. But under current technical level, it is unpractical to obtain 3d face data conveniently with a low cost in real-time, which limits the application of 3d face data in practical face recognition systems.In this paper, we start with 3d face modeling based on stereo images, and then make use of the reconstructed face model, to address face recognition with variant illumination and poses.Stereo calibration is an important stage in stereo reconstruction, the quality of calibration has a direct impact on the quality of reconstructed results. Beginning with the principle of stereo calibration, we analyze the weakness of reprojection error when it is used as the cost function for optimizing the parameters between stereo cameras. Instead, a new stereo calibration algorithm based on rectification error is presented in this paper.Since human face surface is lack of abundant texture features, conventional stereo matching algorithms based on texture correlation cannot acquire reliable disparity result on face stereo images. To solve this problem, this paper proposes a model-assisted face modeling algorithm based on binocular stereo. First, stereo correspondence is reliably performed between input stereo images by employing a 3D reference face model as a medium, then DAISY descriptor is used to further optimize the initial correspondence, and a high-quality dataset of point cloud can be generated by triangulation from the stereo correspondence. Finally, an accurate and feature-preserving 3D face surface is reconstructed from the point cloud dataset by a denoising operation of bilateral filtering and surface meshing.In the study of face recognition with variant illumination, two strategies are proposed to address the problem. The first strategy is to synthesize virtual face images under a variety of illumination conditions by using the reconstructed 3D face model, and establish a multi-illumination face image gallery. Then face recognition with variant illumination can be performed based on eigenfaces algorithm. The second strategy is to train a low dimensional illumination space for human face. For the input face image to be identified, its illumination parameters can be estimated by the illumination space, and combined with the illumination ratio image, a face image gallery under the same illumination parameters can be synthesized from the face gallery under front uniform illumination. Then face recognition with variant illumination can be performed based on cross-correlation algorithm.In the study of face recognition with variant poses, we synthesize virtual face images under 15 poses for each 3D face model by using the reconstructed 3D face model. Then for each pose, a face gallery is established by collecting the virtual face image with the same pose. For the input face image to be identified, first locate the facial feature points, and then estimate the face pose based on these feature points. After that, select the face gallery with the closest pose with input image. Face recognition with variant poses can be finally performed based on eigenfaces algorithm.In the study of face recognition with variant illumination and poses, we synthesize virtual face images under different illumination conditions and variant poses by using each reconstructed 3d face model. Then for each pose, a face gallery is established by collecting the virtual face images with the same pose but under different illumination. For the input face image to be identified, first to locate the facial feature points, and then to estimate the face pose based on these feature points. After that, select the face gallery with the closest pose with input image, and face recognition with variant illumination and poses can be performed based on eigenfaces algorithm.Based on the preliminary solutions, we establish a face reconstruction and recognition system based on stereo images in this paper. The user registration procedure is accomplished by capturing a pair of user's stereo face images and reconstructing his 3D face model. The recognition stage is performed by input 2D images.