Face Reenactment Via Self-supervised Representation Disentanglement

Recent years have seen dramatical advances and widespread use of generative modeling in image synthesis.Face reenactment is a special task in this field,which aims to transfer one's head poses and facial expressions to the face images of another identity,and then generate an image acting as the driving images while preserving the identity of source images.Such ability holds promise to an abundance of applications like image editing,videography,gaming and augmented reality.This task is known to be challenging for two main reasons:one is the intrinsically entangled characteristic of face images,and the other is the low tolerance of human evaluation.To solve these problems,this thesis proposes a novel self-supervised framework(DAE-GAN),which learns to puppeteer talking faces by watching large amounts of unlabeled videos.The main contributions of this thesis are two folds:1.Propose multi-frame/scale deforming autoencoders to disentangle the identity and pose rep-resentations.Only with the assumption of the availability of talking face video sequences,this method can learn to extract pose-free and pose-involving infomation.This manner al-leviates the demand for adequate and accurate face annotations.2.Propose a conditional generative adversarial network which fuses identity and pose from different identities to perform face reenactment task.In addition to the high-quality image generation,this fusion mechanism of generator also makes the few-shot inference possible.When there exists only single or a few source images,this proposed method can still generate photorealistic reenacted faces.The proposed model has been evaluated on VoxCelebl,VoxCeleb2,and RaFD datasets.Ex-perimental results demonstrate the success of face disentanglement and the superior quality of reenacted images between identities.