| With the rapid development and wide application of technologies related to metaverse such as virtual reality(VR)and augmented reality(AR),more and more researchers are focusing on the 3D simulation and reconstruction of 2D space.Nevertheless,most of the existing 3D reconstruction algorithms take the real-world pictures or chromatic images of 2D space as the model input,there are few 3D reconstruction work for the sketch input which is lack of color and more abstract,even if the 3D reconstruction of the sketch,especially the face sketch,is widely used in medicine,criminal investigation,metaverse and other fields,having a wide range of application scenarios.One of the main challenges is that it is difficult to obtain the open-source face sketch-3D paired dataset,which limits the progress of 3D reconstruction from face sketch.To solve the challenges mentioned above,firstly,the dataset of paired face sketch-3D is proposed,so that the 3D reconstruction from face sketch could be realized.The dataset is collected in two ways,which are crossdomain synthesis and manual hand-drawing.Meanwhile,based on the proposed dataset,this thesis explores the implementation of the 3D reconstruction from face sketch under both supervised and unsupervised conditions,and compares the reconstruction results with previous state-ofthe-art 3D reconstruction models,in order to validate the performance superiority of the two reconstruction models under different conditions respectively.The main contributions are summarized as follows.Firstly,a face sketch-3D database is proposed,where the constituent datasets are collected by means of cross-domain synthesis and manual hand-drawing.Among them,the two large-scale datasets obtained by cross-domain synthesis are named by SynBFM and SynCelebA.On the other hand,the two datasets collected by manual hand-drawing are named by HumanAmat and HumanProf,according to the professional level of the hand-drawing sketch.The sketch drawings of HumanAmat are drawn by amateur painters,while the drawings of HumanProf are drawn by professional painters.After that,this thesis explores 3D reconstruction methods of the proposed face sketch dataset under both supervised and unsupervised conditions.For 3D reconstruction under the supervised setting,instead of taking ground-truth 3D faces as supervision information,the supervised method in this thesis utilizes 2D facial landmarks as a kind of weak supervision information to assist the learning process,and utilizes deep perceptual loss function and parametric regularization to improve the reconstruction performance.For 3D reconstruction under the unsupervised setting,this thesis proposes an end-to-end unsupervised learning framework to reconstruct 3D face model from single-view face sketch without 3D face shape template and 2D supervision information.In this learning framework,2D translation module and 3D reconstruction module are cascaded and trained in an endto-end manner,and the intermediate results are optimized iteratively with customized loss functions. |
