Research On Unsupervised Image-to-Image Translation Based On CycleGAN

In recent years,in the context of the rapid development of deep learning and the availability of massive visual data,various visual understanding methods have been developed in the field of computer vision,aiming at assisting machine to understand and analyze the content and semantics of videos and images.Image translation,as a common method of visual understanding,uses neural networks to learn the content of the source domain,and then translate the source domain image into the target domain image.The key is to learn a mapping that can be transformed between different image domains.Many problems in human's production and life can be regarded as sub-tasks of image translation.For example,in the field of medicine,doctors hope to optimize the low-resolution CT generated by X-ray irradiation to obtain high-resolution images for the convenience of diagnosis.In the field of security,the staff want to remove the fog from the images taken on foggy days to make them clearer.In terms of entertainment,people want to achieve interesting goals such as changing dogs to cats,faces to cartoons and so on.With the birth of Generative Adversarial Network(GAN),image translation task has developed more methods with small network and good effect,and the translation work based on GAN has become the mainstream of research.However,due to the limitation of labor and material resources,large quantities of paired training data are often difficult to obtain.Therefore,most of the existing image translation work focuses on unpaired images.The proposal of CycleGAN is a milestone breakthrough in unsupervised image translation,which uses the idea of cyclic consistency to achieve effective generation between two image domains.This network has become the backbone model in the field of unsupervised translation,but it also has the shortcomings of unstable training and poor interpretation of image features.Based on the above background and problems,following works accomplished based on CycleGAN in this dissertation:1.A Reusing feature encoder in image-to-image translation method based on CycleGAN is proposed.The feature encoder used in original CycleGAN applied in discriminator else,it can reduce the number of parameters.During the training,the generator only update the feature transform block and decoder,and the encoder updated by discriminator.Meanwhile,this chapter proposes a feature perceptual loss,which can extract the feature discrepancy adaptively and then constrain the network training.This method achieved better performance on three datasets,horse2zebra,apple2orange and monet2photo.2.An image translation method based on self-attention mechanism is proposed.Using attention mechanism in generator to enlarge the receptive field of the feature map,and enhance its description of the spatial structure around the central pixel.In order to solve the checkerboard artifacts during training,choosing upsampling and conv-layers instead of transpose-conv-layer in feature decoder in generator.At last,the spectral normalization method used in generator and discriminator to solve the problem of slow convergence and instability training.Both quantitative and qualitative experiments prove the effectiveness of the proposed method.3.An image translation method with foreground perception is proposed.In order to solve the problem that the background translated with foreground object,this method in this chapter learns an attention network separately to extract the foreground features of the image,and the learned attention maps force the generator to translate only the target foreground.This method can achieve more accurate foreground generation.