Max-margin Generative Adversarial Networks With Applications To Image Generation And Inpainting

Image generation is a challenging task in computer vision and machine learning.Traditional algorithms studied physical and statistical principles to build generative models,but due to the restriction of model complexity and lack of high-level feature representations,the results of these methods are limited to generate low-level features such as edges,textures and particular patterns.Recent works on representation learning use Deep Neural Networks(DNNs)to construct effective deep generative models,sunch as Deep Belief Networks(DBN),Variational Auto-Encoder(VAE)and Generative Adversarial Nets(GAN).Those works show the strong learning ability and representational capacity of deep moodel.Although they achieve promising progress on natural images generation,the generated samples are still in low resolution,and the quality of generated data is still obviously lower than the quality of real data.The image generation models which learn unconditioned generative models of images still face a lot of challenges.Image inpainting is a technique which aim to fill in missing regions with alternative contents,and make the images look complete and undamaged.Image inpaiting can be apply to recover damaged images or remove unwanted objects in images.Early works extract information from only a single image,they do not capture high-level context of images.Therefore,the completion results of these methods are unsatisfying.To understand the structure of the scene and objects being completed,various DNN methods have been proposed and applied to image inpainting tasks.Generative Adversarial Networks(GAN)have recently received a lot attention due to the promising performance in image generation and inpainting.However,GAN and their variants still face several challenges,including vanishing gradients,mode collapse and unbalanced training between generator and discriminator,which limits further improvement and application of GAN.In this paper,we propose the Max-Margin Generative Adversarial Networks(MMGAN)to approach these challenges by substituting the sigmoid cross-entropy loss of GAN with a max-margin loss.Experiments on four datasets have shown that MMGAN have three main advantages compared with other four models.Firstly,MMGAN is robust to vanishing gradients and mode collapse.Secondly,MMGAN have good stability and strong balance ability during the training process.Thirdly,MMGAN can be easily expanded to multi-class classification tasks.Based on the idea of MMGAN,we proposed a image inpainting model which use both the contextual information and the semantic prior knowledge of images to complete the corrupted images.Two optimization terms(contextual losses and structural losses)and three networks(a generator,a discriminator and a feature matching denoiser)are introduced to push generated data closer to the real data distribution in both pixel space and feature space.Sufficient experiments on CelebA dataset and Cars dataset are implemented to compare the performance of our model with other two stateof-the-art models.The experiment results and analysis lead to a conclusion that our model is effective on image inpainting tasks,and it can handle various types of corrupted images.