Research On Facial Expression Synthesis Based On Generative Adversarial Networks

Facial expression synthesis,as an important technology in image processing,is widely used in film,virtual reality,games,criminal investigation and other fields.Traditional facial expression synthesis algorithms require facial expression acquisition and tracking devices and a lot of computational power to perform complex operations.Facial expression generation adversarial network based on deep learning solves the problems of these traditional methods above,but there are some problems such as synthetic facial expression tearing,blurring,synthesized video expression discontinuity and so on.In this thesis,we study the framework of deep convolution neural network and the theory of generating adversarial network.Aiming at some challenges in the field of facial expression synthesis,we propose corresponding solutions.The main research contents and innovations of this thesis are as follows:(1)For the problems of facial expression image synthesis,such as expression tearing and blurring,we propose the framework of facial expression image synthesis,Face GAN and a structural similarity measure index loss function based on the attention mechanism.Firstly,the facial features of the driving face images are extracted and the facial expression feature map is constructed.Secondly,the facial expression feature map and the original target face image are input into the Face GAN network.In Face GAN,two feature encoders are used to encode the features of the facial expression feature map and the target face image respectively,and these two features are fused to synthesize the facial expression image.Finally,a structure similarity measure index loss function based on the visual attention mechanism is used to calculate the loss between synthetize expression image and corresponding ground truth image to reduce expression tearing and blurring.(2)To solve the problem of discontinuous expression in expression video synthesis,a framework of facial expression synthesis based on recursive dual-generation adversarial network is proposed.Face GAN is used to generate seed images and Fine GAN is used to retain video features.In recursive synthesis,firstly,the deep face feature is extracted and the expression feature map is synthesized,which is used as supervisory signal and Face GAN is used to synthesize the seed image of the facial expression;secondly,the synthesized seed image is used as input together with the original target face image,and the synthesized feature preserves the image in Fine GAN as the output of the current frame,while the feature preserves the image as the next frame.The input of seed image generation;Finally,the next frame image is generated by using serial Face GAN and Fine GAN recursion,and the video sequence of face expression preserving features consistent with the original input expression is obtained repeatedly.Experiments on CK+ and MMI database show that the proposed image synthesis method can synthesize realistic expression images,while avoiding expression tearing and blurring.In video synthesis,the method proposed in this thesis can synthesize clear and natural video frames of facial expression,solve the problem of discontinuous expression or flickering between video frames in the synthesis of video frames of facial expression,and has robustness when the shape of the target face is different from that of the driving emotional feature map.