Cross-Domain Face Synthesis And Application Based On Generative Adversarial Networks

In recent years,with the development of deep learning,image synthesis has further at-tracted much attention.As one of the important branches of image synthesis,cross-domain face synthesis is a valuable research area.It can be described as the transformation among dif-ferent face sets or aiming to find a mapping relationship among face domains.If there is no face image of a subject in the target domain,we can use a cross-domain face synthesis algo-rithm to synthesize it from a domain that has that subject's face image.For example,when tracking suspects,a face sketch drawn with the description of witnesses can be converted into a face photo to facilitate the identification in the face retrieval system.For another case,the thermal faces captured in the night can be transformed into visible photos for recognition.Cer-tainly,users of social network applications may want to convert their portraits into various painting styles for entertainment.There are many application scenarios for cross-domain face synthesis,and one of the representatives is the transformation among face sketches and photos.This thesis proposes to solve the problem of face sketch synthesis,which is also helpful for other cross-domain face synthesis scenarios.Existing methods for face sketch synthesis can be divided into two categories,shallow learning-based,and deep learning-based.This thesis proposes three different models based on deep learning and adversarial training to solve the corresponding difficulties,including how to better preserve identity information,how to obtain a high-performance bidirectional mapping with good image quality and identity preservation,and how to train the model under the condition of lacking paired faces.Experimental results show that this thesis provides effective and practical solutions for cross-domain face synthesis.In summary,the main contributions of this dissertation are as follows:1.We propose an identity-preserved model to solve the problem of how to keep identity information unchanged after the face synthesis.Existing methods prefer to emphasize the qual-ity of the generated image and their evaluations on face recognition are not sufficient or partial.leading the claims for identity preservation is inappropriate.We thus propose three identity preservation strategies for face sketch to photo synthesis,which can enhance the identity in-formation during the transformation.The experimental results demonstrate that the proposed model can significantly improve the face recognition accuracy after synthesis,which indicates the capability of identity preservation of the model.2.A feature injection method is proposed to revise the output from the middle layers of a cross-domain face synthesis network to improve the generation.If a face synthesis network cannot effectively extract the input face features from different convolutional layers,it will cause identity information of the final synthesized face lost with error accumulation.On the other hand,the features from different middle convolutional layers of a well-trained face feature extractor contain different levels of identity information,so these auxiliary features can be used to revise the results of the training network.Experimental analysis and results show that we can use a high-performance off-the-shelf feature extractor to extract multi-level features from the input image,and then add them to the corresponding output of our network layers so that it can improve the face generation.3.We propose a bidirectional mapping model based on deep feature injection to obtain a simultaneous translation between face sketches and photos,which has both good image quality and identity preservation.Existing methods are difficult to complete the bidirectional mapping well,preferring to be effective in a unilateral mapping,which may cause performance degrada-tion for the reverse processing.This may be due to the large differences between two domains,so a mechanism specially designed for one input domain may not have the same effects in another one.Therefore,we propose a light model using the interpolation method and small convolutional filters.It adopts a shared encoder to reduce the domain-specialized information and employs two separate decoders to simultaneously complete the transformation.The exper-imental results suggest that our approach is superior to existing methods in synthesizing both sketches and photos,regardless of identity preservation or image quality.4.An unsupervised learning method with feature injection is proposed to the problem of lacking paired faces.At present,supervised learning methods are still the mainstream in cross-domain face synthesis,but it is difficult to collect paired faces in practical applications.Therefore,based on unsupervised image-to-image translation,we propose a multi-domain face synthesis method.We introduce two general constraints and utilize the feature injection to solve the problem of unsupervised face synthesis and enhance its generalization ability.The experimental results demonstrate that the visual quality of the sketches and photos generated by our method is comparable to that of state-of-the-art supervised face sketch synthesis methods,and is also superior to existing unsupervised learning methods.