Heterogeneous Face Synthesis Via Generative Adversarial Networks

Heterogeneous face synthesis(HFS)aims at generating realistic and recognizable face portraits in a variety of modalities,such as sketches,comics,etc.HFS has a wide range of applications in both digital entertainments and criminal investigations.Recently,researchers have proposed a lot of HFS methods based on generative adversarial networks(GANs).However,there is no systematical analysis about these works.In addition,existing methods are sensitive to poses and lighting conditions,and cannot produce realistic heterogeneous images for faces in-the-wild.According to these circumstances,the contributions of this paper are briefly summarized into the following two folds:Firstly,we briefly review the history of HFS.Besides,we summarize the latest developments from various issues,including the applications,model architectures,performance evaluation,datasets and qualitative analysis.Finally,we forecast the challenges and potential trends in this area.This work is instructive for those who want to quickly understand the status of HFS and to determine their research topics.Secondly,we propose a structure-adaptive robust face sketch generation method.Since textures in a face sketch are highly correlated with the semantic information,we use face parsing masks to conduct spatially adaptive normalization,so as to guide the texture synthesis in different facial regions.Besides,we enforce the generator reconstruct the face parsing masks,by using a novel structure-reconstruction loss.This loss makes the synthesized sketch structurally consistent with the input face photo.Finally,the ground-truth photo-sketch pairs are not pixel-wise aligned,which might hurt the performance.We therefore propose a slack reconstruction loss.Experimental results on multiple datasets show that our method can produce high-quality sketches,even for faces with extreme poses or lighting conditions.In conclusion,our method can robustly produce sketches with distinct structures and realistic textures.This work is meaningful for both the theory and applications in this field.