Facial Representation And Animation Towards Digital Human Applications

Constructing digital twins for real humankind,i.e.,the vivid virtual human,has always been a concerned topic lying in the interaction area of computer graphics,com-puter vision and multimedia community.This dissertation will focus on the facial repre-sentation and animation problem towards digital human applications,and individually introduce our research progresses within these areas.Our research is targeted for the facial-related tasks,including the facial identity and expression representation,facial retargeting and cross-modal facial animation.Because the human faces have compli-cated natural mechanics and powerful social interactive functions,it is important to properly represent the facial expressions for the down-stream applications.In order to model the abundant facial expression semantics within computational devices,we first propose a deep learning based framework for facial expression representation learning.The framework takes either 3D face shapes or 2D facial images as input.By disen-tangling the identity and expression component,our method is capable of representing the pure expression information with continuous distributions.Based on the novel fa-cial representation,we further explore the facial retargeting task between human and avatar.Compared to the traditional artist modelling pipeline,which suffers from time-consuming and high expertise requirements,our method is developed by engaging or-dinary people into the facial expression similarity judgement process.In this way,our approach can significantly reduce the expensive cost,both in time and expertise aspects,and produce high-quality cartoon animation sequences.Besides,we also extend the ex-pression transfer task to other objects like caricatures.With help of 3D caricature recon-struction and texture generation,we propose a method which is capable of animating the exaggerated caricature faces with expressions extracted from normal faces.To achieve this target,we combine deep learning techniques and geometric optimization methods into a single framework.We conduct both quantitative and qualitative evaluations on all the mentioned works.The experimental analysis proves that our methods have the superior performances to the previous ones,either in terms of novelty or in terms of applicability.We believe this series of works would benefit the following research and motivate the digital human applications in the future.