Face Transformation And Editing Based On Generative Adversarial Networks

Face image transformation and editing is one of the most promising directions in the field of computer vision,and it plays an important role in entertainment,industrial production and law enforcement.In recent years,deep-learning methods based on generative adversarial networks have greatly improved the quality of image synthesis,and have also developed many applications in face transformation and editing,among which photosketch synthesis and recognition and facial expression synthesis belong to the most representative work in image transformation and face editing respectively.However,there are still some shortcomings in this field.On the one hand,the synthesized images are indistinct and has many distortions.On the other hand,the generation process ignores the face features when considering recognition problem,resulting in the loss of facial fidelity.In order to solve the above problems,this paper studied on the two directions based on generative adversarial networks:face photo-sketch synthesis and recognition and facial expression synthesis.The corresponding research results are as follows:1.On the basis of CycleGAN,to solve the problem of losing identity information during the generation process,Identity-Aware CycleGAN(IACycleGAN)is first proposed using a new perceptual loss to supervise the image generation network which makes the model focuses more on the synthesis of facial feature regions.In addition,a mutual cyclic optimization method between the synthesis and the recognition network is proposed.The results show that the proposed method not only improves the quality of the synthesized images,but the recognition accuracy on the CUFSF dataset is also greatly improved by 21%.2.In order to make the convolutional layers focus more on the expression information of the images and solve the problem of generating blurred expression details,Attention-Fused Generative Adversarial Network(AFGAN)is proposed,which combines two attention mechanisms(the mask attention and channel attention)with the face synthesis,and the idea of multi-scale discriminators was adopted during training as well,which distinguish between real and fake samples for different expressions.In addition,the identity loss is proposed to make the generation of the content mask depend on the input images,which greatly improves the overall quality of image fusion.The qualitative and quantitative analyses on the RaFD dataset have verified the effectiveness of this model.