Improved Generative Adversarial Networks For Image-to-Image Translation

Image-To-Image Translation(I2IT)aims to transform images from a source domain to a target domain.Recently,Generative Adversarial Networks(GAN)have achieved great success in various I2 IT tasks.However,it is still challenging to generate high-quality image details and textures.Existing methods typically neglect the significance of image composition in guiding the translation process.Besides,they merely use a single or unique generator architecture,which always fail to capture multi-scale and multi-level structures in an image.To combat these challenges,in this paper,we propose a composition-aided GAN for face photo-sketch synthesis,and a heterogeneous GANs for universal I2 IT tasks.The contributions of this paper are mainly two-fold:First,we propose a Composition-Aided Generative Adversarial Network(CA-GAN)for face photo-sketch synthesis.Considering the specific structure of human face,we first use facial parsing masks as auxiliary inputs to help the transform of face photo/sketch.Furthermore,we use the perceptual loss to enforce the synthesized image preserving the identity information of the given image.Finally,we use a stack of GANs to further optimize details,and empirically analyze the stacking strategy.Experimental results show that our method can synthesize face photos/sketches with realistic textures and preserved identity,as well as outperform existing methods,over a wide range of challenging datasets.Second,we propose to use complementary,heterogeneous adversarial networks for universal I2 IT.Specially,we use a deep U-Net and a shallow residual networks as generators,respectively.These two generators are different in network architectures and naturally perform in different scales and different positions.Afterwards,we use a gated fusion network to fuse these two generators and produce an output.Here,a gate unit is used to automatically weight the significance of these two generators.Finally,we propose a multi-level integration discriminator that integrates multi-level and multi-scale features to improve the generator's ability to generate high-quality details.Quantitatively and qualitatively evaluations on various datasets show that our method significantly improves the quality of generated images across various I2 IT tasks.In summary,we propose two improved GANs and apply them to various I2 IT tasks.The proposed GANs considerably improve the quality of generated images and outperform existing methods.The work is meaningful for both the theoretical research and practical application in this area.