3D Face Reconstruction And Interactive Free Viewpoint Video Generation

3D face reconstruction and free viewpoint video have broad application prospects in the fields of film,games,animation and sports events,respectively.The key to their large-scale landing application is to efficiently generate high-quality results.The traditional methods for high-precision 3D face reconstruction and free viewpoint video generation mainly face the following challenges: 1)the requirements of a large number of high configuration hardware equipment,2)the generation process is complex and the accuracy depends on the precise results at each step,and 3)the generation process is time-consuming.The present study aims to improve the efficiency and quality of 3D face reconstruction and free viewpoint video generation using advanced deep learning methods.To provide a better database for deep learning method,a high-precision 3D face database is constructed using traditional multi-view 3D reconstruction method.The main contributions of the dissertation include:1.Propose a scheme of high-precision 3D face database construction and registration,including multi-view image data acquisition,high-precision 3D face reconstruction,geometry and texture registration.Before registration,the geometry is hierarchically represented as low-hierarchy 3D face and displacement map,which improve the efficiency of model storage.2.Propose two methods of low-hierarchy 3D face reconstruction from a single-view image.The first is a kind of fitting reconstruction method by constructing a bilinear parametric model,which is evaluated to outperform the other two public latest parametric models in fitting ability.The second method predict 3D face based on graph convolution neural network,which improves the reconstruction speed.In addition,two kinds of generative adversarial network are designed,realizing the texture complement and displacement map prediction,which supplement the medium-high frequency geometric details.3.For free viewpoint video generation,two virtual viewpoint image generation algorithms are proposed.Two sets of multi-view synchronous video acquisition systems respectively for high frame rate and high resolution are designed and constructed.The collected data were used for experiments,which compare with other algorithms in qualitative and quantitative metrics.The experimental results show that the proposed virtual view image generation algorithm based on deep learning can greatly improve the speed while ensuring the quality of the generated image,and the method of 3D reconstruction and re-rendering is more accurate in complex scene area but timeconsuming.Finally,the interactive software based on Unity is also designed for free viewpoint video.