3D Face Modeling Based On Multiple Photos

In recent years, with the development of computer graphics and computer vision technology, 3D face modeling technology is also developing rapidly, it is widely used in video conferencing, virtual reality and computer games etc. However, because of the complexity of human face, creating personal 3D face model is a persistent challenging task.After reading a lot of relevant literature at home and abroad, this paper comprehensively analyzes the advantages and shortcomings of previous research results, especially 3D face modeling approach based on multiple photos, we propose an improved method for key step. If the number of their feature matching(recovered 3D points and SIFT feature points of recovering photo) is scarce(less than 16), we propose an improved algorithm, epipolar constraint and ZNCC algorithm are used to satisfy initial condition of DLT algorithm.The main content is as follow:(1) Feature detection. In this paper, photos are taken in five different angles from left to right. The localization, orientation and descriptor of feature points are computed by using SIFT algorithm for every photos.(2) Feature matching. Constructing and searching algorithms of k-d tree are used to compute the initial matching results on the adjacent two photos.(3) Recovering camera parameters and reconstructing sparse 3D points. Firstly, based on the results generated in the previous step, fundamental matrix is calculated by using RANSAC eight algorithm to retain more extract matching results. Secondly, RANSAC five point algorithm is used to recover camera parameters of first two photos, and the triangulation algorithm is used to reconstruct sparse 3D points cloud. Finally, camera parameters of the rest of photos are recovered using RANSAC DLT algorithm, and the new sparse 3D points cloud are reconstructed using triangulation algorithm. Camera parameters and 3D points cloud are optimized by using SBA algorithm.(4) Deforming generic model. In this paper, 3DMM is used as generic face model. Firstly, ICP algorithm is used to establish corresponding relationship between 3DMM and sparse 3D points. Then the parameter α of 3DMM is solved using LM algorithm to reconstruct 3D face geometric model. Finally, multi-resolution technique is used to synthesize facial texture image. After texture mapping, the personal 3D face model is built.In summary, this paper has optimized and improved key steps of 3D face modeling method based on multiple photos. Experimental results show that our method can generate realistic 3D face model.