A Face Reenactment Algorithm Based On Flow Field Decoupling

Face reenactment is a face image generation method.Its main task is to generate a new image given a source image and a driving image,which has the facial expressions and head poses of the driving image while retaining the content information of the source image.It has a wide range of application scenarios such as image editing,entertainment,human-computer interaction,cultural education and so on.Therefore,face reenactment has received more and more attention and research.The face reenactment task involves two major points,one is the representation and extraction of facial motion,the other is image generation.Existing face reenactment algorithms based on flow field deformation extract the complete facial motion flow field from the source image and driving image,and cannot partially control the facial motion,such as only transforming expression or only changing posture,lacking flexibility and controllability.In addition,in the face reenactment method based on flow field deformation,the process of extracting motion flow field does not consider character identity information,resulting in the existence of identity information in the process of image generation the problem of leakage causes a certain loss of the identity information of the characters in the generated images.Aiming at the above problems,this thesis proposes a face reenactment algorithm based on flow field decoupling.The main research contents are as follows:1)Using the characteristics of two different facial movements of expression and posture,a loss function is designed to decouple the two movements,increasing the flexibility and controllability of the face reenactment method based on flow field deformation.Flow field deformation is to characterize the motion of pixels in the image by means of optical flow,and then sample the source image to obtain the generated image.In the process of face reenactment,the motion flow field caused by the change of expression is complex but small,and the motion flow field caused by the change of posture is large but consistent.Design a loss function based on this characteristic to realize the separate control of expression and posture,and increase the controllability of the algorithm.In addition,there is in-plane motion in the process of flow field deformation,which leads to information loss.In this thesis,a collision prediction network is designed,and image inpainting technology is used to repair the sampled image,reducing the information loss of the image and generating more realistic results.Compared with other face reenactment algorithms based on flow field deformation,the algorithm in this thesis can increase the controllability of the algorithm without losing the effect of face reenactment.2)Use the training strategy of curriculum learning to alleviate the problem of identity leakage in the process of face reenactment.The deformation-based face reenactment method represents motion in the form of a flow field,and does not consider the interference of identity information on the flow field when acquiring motion information.The identity information such as the facial contour of the driving character is transferred into the reenacted image,resulting in the distortion of the face reenactment result.In response to this problem,this thesis adopts the training method of curriculum learning to alleviate the identity leakage problem in the face reenactment method based on flow field deformation to a certain extent.3)Using face recognition,expression detection,and gesture detection networks,objective evaluation indicators are designed to objectively evaluate the effect of face reenactment.Since most of the existing method evaluate the result by recruiting volunteers or distributing questionnaires.There are certain subjective factors in this evaluation method.In order to evaluate the reenactment results more objectively,this thesis uses the existing relatively mature identity,expression,and posture detection algorithms,and designs CSIM,ED,and PSIM objective indicators to evaluate the replayed results,so that the evaluation results are more objective.