Image Style Transfer Via Deep Learning

Image style transfer aims at transferring a content image to a target style without changing the content.The corresponding tasks such as image super-resolution reconstruction and face photo-sketch synthesis,have a wide range of applications in the public security and digital entertainment areas.Recently,deep learning techniques have achieved great success in various based image style transfer tasks.However,it is still challenging to efficiently train a deep network to generate realistic details.Besides,existing methods typically produce complicated textures in generated images,thus limiting the application of such techniques.To combat such challenges,in this paper,we propose a novel generative adversarial networks(GANs)and a novel face sketch synthesis algorithm.The contributions of this paper are mainly two-fold:(1)Firstly,we propose an Incrementally focal loss(IFL)GAN for image-to-image(I2I)translation and image generation.In IFL-GAN,we propose a novel IFL function,which makes the model gradually focus on difficult examples during training.This would improve the power of the model in generating details.In addition,we propose an enhanced self-attention(ESA)mechanism to improve the representation learning ability of the generator.We apply both IFL and ESA to unsupervised and conditional GANs.Experimental results show that the improved GANs significantly improve the quality of generated images in various I2 I translation and image generation tasks.(2)Secondly,we propose a robust face sketch synthesis algorithm.Specially,we first use a component sparsity constraint to make the model merely produce primary boundaries of a given face.In this way,the synthesized sketch would contain a small number of textures or lines.In addition,we propose a self-consistency loss,which makes the model to reconstruct style images.The self-consistency loss proves to improve the continuity and realism of the lines in the sketch.Finally,we propose a number of pre-and post-processing techniques to refine details and the background.Experimental results show that this method can transfer a given face photo to a realistic sketch,which composes a sparse set of continue brush-strokes.In addition,this method generalized well to faces with varying and backgrounds as well as universal images.This is meaningful for enriching the practical applications of imagestyle transfer techniques.To sum up,we propose two novel image style transfer methods in this paper.Compared with existing works,our methods significantly improve the quality of the generated images.The proposed techniques can be extended to other tasks and are significant for the practical applications in the image style transfer area.