Image Synthesis Based On Generative Adversarial Networks

Image synthesis is an important research direction in the fields of computer vision and computer graphics.It has a wide rang of applications:image generation from a piece of text,image translation between different domains,image restoration,editing,deblurring,super-resolution,etc.Despite decades of research,the performance of image synthesis models is still not satisfactory in the face of complex natural images.The main challenges in synthesizing images are realism,diversity,and consistency with input conditions.The emergence of generative adversarial networks(GANs)in recent years has increased the realism of generated images.However,the challenges in image synthesis still exist due to problems in GANs such as:unstable training,inability to judge the convergence state,and mode collapse.The core contributions of this thesis are to propose some solutions to the challenges in image synthesis and problems in GANs.This thesis proposes a feature matching loss function to solve the problem of un-stable training in GANs.In the training stage,for the discriminator network,we use the same binary cross entropy loss function as in the original GANs to maintain discrimi-native ability.For the generator network,we use the loss function of feature matching,which requires the feature center of generated image to be close to the feature center of real image,thus solving the problem of gradient vanishing in the original loss function of GANs.The training of GANs is more stable.The experimental results show that the loss function effectively solves the instability problem in the GANs,and improves the quality of the generated images.This thesis proposes to add the encoder network to the GAN framework to solve the problem of mode collapse in GANs.The encoder network maps the image space to the latent space,and then uses the generator network to map the hidden space back to the image space.Because the images in real image space are diverse,the images generated by the generator network are also diverse.This solves the problem of model collapse in GANs.The experimental results show that the GAN framework with encoder generates more diverse images,which proves that the framework effectively solves the problem of mode collapse.At the same time,the framework can be applied to many applica-tions:fine-grained image synthesis,image inpainting,image morphing,image attribute retrieval,data enhancement,and so on.This thesis proposes identity preserving GANs framework to solve the problem of open-set identity preserving face synthesis.The framework can disentangle the identity features and attribute features(poses,expressions,illuminations,etc.)in the face im-age,and then recombine the identity features and the attribute features extracted from another face image,and input them into the generator model and get a new face image.The face image maintains the given identity feature while also maintains the given at-tribute feature.The experimental results show that the framework can perform open-set identity preserving face image synthesis.At the same time,the framework can be ap-plied to many tasks:profile face to frontal face,adversarial examples detection in face recognition system,face attributes translation,and so on.