| 3D face reconstruction is a classic computer vision problem with broad applications,such as face recognition,virtual reality,and face model design.3D face reconstruction from photos has attracted much scholarly attention,but few scholars have paid attention to 3D face reconstruction from other portrait forms.Photo shooting is easily affected by factors such as light,making it sometimes difficult to capture certain details of faces.However,sketches are created by artists,which can emphasize the characteristic features of faces.Therefore,this paper conducts research on 3D face reconstruction from sketches,the aim of which is to reconstruct 3D face models with realistic overall shapes and characteristic features.To alleviate the lack of paired sketch-3D training data,this paper introduces the photo-to-sketch synthesis technique to obtain synthesized sketches.With the help of available paired photo-3D data,this technique can build a bridge between sketches and3 D data.Besides,a dual-path network is further designed to support the collaborative training of photos and sketches.The dual-path network can encode photos and sketches into the corresponding style space and content space,where the style space encodes the image style and the content space encodes the face shape.The dual-path design allows3 D reconstruction from sketches can make use of the supplementary information in photos.Comparative experiments prove the superiority of the dual-path network,which can reconstruct a convincing face model with certain characteristic details.To further improve the reconstruction effect of characteristic features on the face model,this paper introduces a line-guided feature refinement technology based on the dual-path network.A two-stage neural network is designed,where the first stage focuses on the reconstruction of the overall face shape using the dual-path network,andthe second stage focuses on using lines to guide the refinement of characteristic features.Specifically,characteristic lines are extracted from the input sketch,and a line extraction module is developed to simulate the same line extraction operation from the reconstructed face model.A novel line loss function is then proposed to compare the two extracted lines to guide the refinement of the face model.Experiments show that the method in this paper can reconstruct realistic face models with characteristic details,and demonstrate that the method supports the editing of face models,that is,adding or removing lines to edit the details of face models.The dual-path design enables the 3D face reconstruction from sketches to benefit from the supplementary information in the photos.In addition,this paper can restore the characteristic features emphasized in sketches but not emphasized in photos.Our work proves that different portrait forms can promote each other in face modeling,which provides inspiration for multi-modal learning in the field of 3D face reconstruction. |
