Research On Face Photo-sketch Transformation Based On Improved Generative Adversarial Networks

Using sketches is one of the main technical means of search and arrest of suspect in the process of criminal investigation.Face photo-sketch transformation is aimed to synthesize the given sketch(or face photo)into high-quality face photo(or sketch),then to match with the image in the photo(or sketch)database find out the identity of the person to be recognized or narrow the scope to be recognized.To improve the quality of images synthesis in the process of face photo-sketch transformation and lay the foundation for further improving the accuracy of face recognition,the following works are done in this thesis:(1)The effects of various loss functions are analyzed,and the influence of different configurations of loss functions on residual generative adversarial networks is verified by experiments.Based on the above experimental analysis,the combination of the adversarial loss and the perceptual loss is used as the basic loss function configuration of model training and the residual generative adversarial network is improved for different problems.(2)The thesis proposed an algorithm for face photo-sketch transformation based on self-attention residual learning.This method,firstly,use down sampling block to extract the shallow features of the input image.Secondly,the deep features of the image are encoded by residual blocks embedded with self-attention mechanism to adapt to the weighted feature statistics and selectively emphasize the important information.Finally,the self-attention features are reconstructed by up sampling block and are further enhanced by global residual mechanism,then to generate the final face sketch or face photo with the same resolution as the input image.Simultaneously,the multi-scale gradients technology is used to ensure the quality of synthesized image and avoid the influence on the model caused by the instability of GAN during the training.Experimental results show that the proposed method can improve the quality of image synthesis in the target domain both in face photo-sketch synthesis and face sketch-photo synthesis,and it is superior in evaluation criteria of SSIM,FSIM and face recognition accuracy.(3)A method for face photo-sketch transformation based on unsupervised style encoding is proposed in the thesis.To begin with,the content image(face photo)is encoded as the content feature vector by the Content Encoder,and the style image(face sketch)is encoded by the Style Encoder.Further,after a series of pooling and full connection calculation,the style parameters are added to the bottleneck block of decoding,and the target style image(synthesized sketch)is reconstructed by the Decoder.In addition,due to the decomposition of content and style space,this framework can perform style-guided image-to-image transformation,where the style of the converted output is controlled by the sample image in the target domain provided by the user.The effectiveness of the proposed method is verified by comparing with the synthesis results of several state-of-the-arts of sketch face synthesis.