Sketch-to-Image Translation By Using Generative Adversarial Networks

With the development of deep convolutional neural networks,generative adversarial networks have been applied in the field of image translation and have achieved excellent results.The translation from sketch to image(S2I)is to reconstruct other useful information from the sparse information of the sketch image.It is great significance to research S2 I,however,because sketches can only provide sparse information,and lack the color and advanced semantic information to improve image quality,it has always been a major problem in image translation.This paper believes that the translation of human face sketch images not only requires the reconstruction of global structure and local texture information from ordinary images such as flowers and cars,but also requires accurate translation of local attributes to improve the visual effect of human face images.Therefore,The task of the translation from sketch to image is divided into general sketch and face sketch two branches.The difference operator is used to process the color image to obtain the binary image boundary to simulate the sketch.The dynamic difference operator parameter increases the type of sketch,improves the generality of the network and prevents over-fitting to a particular style of sketch.For translation tasks of general sketches,this paper introduces a total variation loss based on the Pixel-to-Pixel(Pix2Pix)image translation network to constrain the reconstruction of local information and improve the translation quality.Aimed at the characteristics of face sketch,this paper proposes a new image translation algorithm: face sketch image translation network under the constraints of text attributes.The core of the algorithm is to introduce text attribute constraints,and use the method of perceptual fusion to combine the sketch with the corresponding text.Under the constraints of improved pairing generative adversarial loss and introduced small area to correcting pixel to pixel loss,the network completing the task of translation from face sketch image and color face image.Based on FID(Fréchet Inception Distance,FID),this paper introduces two algorithms of face similarity comparison and face attribute analysis in the field of face detection as evaluation matrices to evaluate the global translation effect and local attribute translation effect in the face sketch image translation.The training and testing results on different data sets show that the two sketch translation networks mentioned in this paper both improve the quality of sketch image translation and show better visual effects than related algorithms.