3D Morphable Face Reconstruction And Real-time Expression-driven System

Facial expression-driven is to control a virtual model following facial movements of a performer,achieving similar expression effects.It is widely used in the fields of movies,games,virtual anchors,conversational agents and so on.The process of expression driving includes virtual model making,expression recognition and expression animation.Among them,realistic virtual model making requires 3D reconstruction of a real face in advance,and involves complicated manual expression editing,which limits the instantly use of realistic face models in real-time expression-driven systems.Thus,existing systems usually use pre-prepared 3D characters with multiple expression basis instead.The key technologies involved in expression-driven process are explored,and an automatic system integrating deformable face model reconstruction,expression recognition and expression animation is designed and implemented.The system consists of two data channels.One is to input a single face RGB image and output a deformable face model with expression basis.Second,facial video is used as the input and captured expression parameters is transferred to control the reconstructed face model outputting facial expression animation similar to the input.The main work of this project includes: improving the existing face reconstruction network model based on deep learning in reconstruction speed,applying deformation transfer and expression fusion algorithm to the reconstruction results,and realizing automatic deformable face reconstruction based on single image input;Integrating morphable face reconstruction,facial expression recognition and expression animation jointly to achieve real-time facial expression-driven.Experiments show that the improved morphable face reconstruction technology is better in speed and avoids a variety of manual operations of the traditional process.The expression-driven system achieved expected effect and is of low cost,easy transplantation and automation.