High-Fidelity 3D Face Reconstruction And Animation Research

3D Face reconstruction and animation has been a hot research topic due to its many applications in film making,video broadcasting,online interaction,etc.The requirement of costly 3D reconstruction and motion capture apparatus as well as the labourintensive rigging process limits the application of traditional high quality facial animation technics used in film industry.To tackle these issues,we focus on high-fidelity 3D face reconstruction and animation based on monocular images.Our contributions can be summarized as follows:1.We improve facial registration method and process a large-scale high quality 3D face dataset,which is used to create 3D face morphable model that has stronger representation power than previous works.By employing the consistency in UV texture space of different expressions,we propose to refine the registration of traditional nonrigid ICP algorithm based on optical flow in UV space,which facilitates our high quality bilinear model.2.We propose the dynamic detail synthesis algorithm to reconstruct detailed riggable 3D face from a single image.The high frequency details of raw scans in our dataset are represented as displacement maps.By learning from the dataset,we model the relationship of identity,expression and 3D details with deep neural network and predict dynamic displacements from texture maps and expression parameters.Our predicted 3D models are riggable with highly detailed geometry under different expressions.3.A full system of image animation is presented.On top of the bilinear model fitting method,we incorporate a hair depth prediction network trained on video dataset in a self-supervised manner and generate the warping field based on target rigging parameters.Another network is used to refine the artifacts and synthesis expression related details.Our system can generate high quality animation videos and is robust to large pose variation thanks to the 3D based warping framework.