3D Parametric Model Guided Photo-Realistic Facial Expression Synthesis

Facial information is the main way of human emotion expression.In the past 30 years,scholars in the field of computer vision have carried out a lot of researches on the related topics of face information processing,and achieved fruitful results.As an important research topic in this field,facial expression synthesis algorithm has been widely investigated.It has a wide range of application scenarios in the fields of humancomputer interaction,film special effects,virtual reality,animation character production and so on.Although great progress has been made,the current facial expression synthesis methods still have some certain limitations and defects,especially in the synthesis of continuously tunable facial expressions.There are few research results on the different intensities of the same expression generation,and the synthesis effects of existing algorithms are also unable to reach the application requirements.After analyzing the current mainstream facial expression synthesis algorithms,this paper proposes a novel algorithm framework,which can synthesize facial expressions with tunable intensity based on a single face image.Seven different facial expressions could be synthesized,including surprise,fear,disgust,happiness,sadness,anger and neutrality,and the intensity of each expression is tunable,such as from smile to laugh.Specifically,the framework proposed in this paper firstly generates the face parametric model with different expression and intensity,and then uses the rendered 3D model to synthesize photo-realistic face images through Generative Adversarial Network(GAN).The main contributions of this paper are summarized as follows:1)In this paper,we propose a novel framework to synthesize photo-realistic facial expression images,and an anto-encoder network combined with 3D parametric face reconstruction is designed to generate 3D facial expression models with different intensities.2)A variety of loss functions and network structures based on GAN are designed, and photo-realistic facial expression images are synthesized from the input 3D rendered face model through this framework.3)The experimental results on public datasets show that the proposed algorithm can synthesize natural and photo-realistic seven facial expressions with tunable intensity from a single face image.The synthesis effect is better,and it has more abundant detail features.It surpasses other most advanced algorithms in term of image definition and similarity with input face.Finally,the experimental results show that the algorithm framework can be derived and applied to face changing and other fields.And the 3D face information is used for expression recognition,and the accuracy rate reaches the state of the art.