X-Face:Disentangled Representation For Face Image Synthesis

As the rapid development of image generation based on deep learning in recent years,face images have become the central technology of image generation as the abundant application can be applied in.Customized face generation has a wide application territory,such as face swap,customized age or gender,which have already been widespread practice in socializing and entertaining.Meanwhile,face reenactment undertakes the growth of emerging technology such as virtual anchor,dynamic head photo.In this paper,a new face generation algorithm based on generative adversarial network is proposed,which can separate,the identity,expressions and environmental attributes of the face.By controlling and recombining the three separate representations,a highly controllable customized face generation will be possible,single model can support both face swapping and face reenactment applications.The algorithm framework is divided into two parts:face attribute extraction and fusion generation.The first part can extract identity vectors,face landmarks and environment vectors from the input face images.The second part takes three attributes as input to generate a new face.In video face swapping application,face landmarks and environment vectors are provided by faces in video frames,while identity vectors are provided by single target face,so that in face swapping video,the new face with the target face identity while maintaining the expressions and environment of the video character can be generated.In face reenactment,the facial expression and pose can be accurately controlled by adjusting the input of landmarks.In order to implement face generation based on disentangled representation,we designed a controllable fusion generation network framework based on generative adversarial network.Meanwhile,we further studied the supervision in the generation task,designed a training method of cyclic constraints,which allows the network to spontaneously extract the learning environment attributes and fuse the face generation without additional labeling.We have done a series of comparative experiments and comparisons on the Faceforensics++[1]dataset to further illustrate the face generation performance of this method.