Face Reconstruction System Based On Multiple RGB Images

With more and more user-centered products booming recently,user-generated content has become a popular trend in the Internet,which focuses on the establishment of user avatar.Meanwhile,the face reconstruction technique is the most complicated part in the establishment of user avatar,and can determine the quality of final avatar result.Since smart phones have been widespread,how to reconstruct high-quality face models via several selfies taken by users has become an urgent problem under the trend of user-generated content.This paper aims to design and implement a face reconstruction system based on multiple RGB images,which can reconstruct high-quality face models according to selfies uploaded by users.The system includes three modules:highlight removal,face geometry reconstruction and face texture reconstruction.And our work is as follows.Firstly,in terms of the excessive highlight removal and color distortion issues of traditional algorithms,we apply pixel classification and compensation function to improve the performance of joint bilateral filter based algorithm and eliminate the black spot phenomenon.Secondly,in order to generate a high-quality face geometry,we propose a coarse-fine double-grained face geometry reconstruction,which takes use of 3d morphable model and shape from shading to restore the global face shape and local details respectively.Thirdly,aiming at the misalignment between multiple images,we use the patch based texture mapping to solve the problem of texture blur.Furthermore,due to the inefficient PatchMatch,we use the KDTree to optimize the search space and get at least 5 times faster.Finally,a complete face reconstruction system is constructed by organically combining these three modules with parameter control and visualization.