Research On High-fidelity Facial Expression Dynamic 3d Recconstruction And Retargeting

With the development of digital human technology,facial animation technology has been widely studied by researchers.The performance-driven animation methods have higher accuracy and better realism,so they are widely used in the film and game industries.This study proposes a series of algorithms to achieve dynamic reconstruction and migration of face expressions with high realism in terms of face modelling accuracy and model-driven smoothness and fineness.The main content of this study can be divided into four aspects to achieve high realistic face expression reconstruction and retargeting:(1)In this thesis,a system based on near-infrared high-frequency binary structured light technology reconstruction techniques was proposed to capture dynamic objects in real-time.The infrared structured light system adopts Gray code plus line shift as the coding strategy,which can achieve sub-pixel localization and thus obtain high accuracy3 D measurement results.Furthermore,to further improve the reconstruction accuracy of our system,we propose a calibration method to find the optimal calibration parameters of the obtained structured light system.To enable the structured light system to work at the maximum frame rate of the hardware,we parallelize the decoding algorithm on the GPU device.This work lays the foundation for the subsequent dynamic 3D facial reconstruction.(2)In this thesis,a 3D facial acquisition system using the proposed high-precision structured light reconstruction method have been built.Then,this setup has been used to collect two 3D facial expression databases with high-precision and high resolution.Compared to state-of-art 3D facial datasets,our dataset has the advantages of rich metadata,rich expressions,and large samples.(3)The performance of facial animation is highly correlated to the acquisition device.Based on the work of near-infrared structured light technology mentioned above,this paper proposes a system and associated algorithms for high-fidelity 3D real-time face animation generation.These methods include the GPU-based non-rigid alignment method,optical flow-based model tracking algorithm,texture fusion algorithm,etc.This facial capturing system we built is easy to deploy,and it can obtain facial expressions with high precision and high fidelity.Furthermore,the system can capture real-time high-precision and dense facial expression movements of human faces.(4)The state-of-art face dense facial retargeting methods can only work well on glossy smooth face sequence models.Therefore,this thesis proposes a non-rigid registration algorithm based on key point deformation to warp the input non-smooth non-regular expression sequence into a uniform topological triangular network.Finally,we adopt an optimation function with two constraints to obtain accurate and robust results.Besides,several experiments have been conducted to verify each chapter's proposed algorithms' performance.This thesis proposes a series of related devices,algorithms,and data for the dynamic 3D facial reconstruction and retargeting algorithms.This work can provide a new solution for the game and movie industry.