| Based on class labels or other auxiliary information,image generation aims to build a generative model to produce new images with desired semantic content.Image gen-eration is one of the fundamental research topics in computer vision and graphics re-searches.It has been widely adopted in many applications,including raindrop removal from images,image reanimation,face replacement,image translation,image super-resolution and so on.In recent years,the two prominent generative models are Varia-tional Auto-Encoders(VAEs)and Generative Adversarial Networks(GANs).VAEs and GANs have achieved promising performance in a wide range of tasks.However,despite the success,they still have their own disadvantages:the generated images of VAEs of-ten tend to be blurry.As for GANs,it still suffers from some training problems,such as unstable training and mode collapse.Also,some generative applications,for exam-ple face replacement method(e.g.,DeepFakes),could be misused for malicious purpose,causing privacy security problem.In this thesis,our main contributions are to propose some solutions to the robust-ness problem in generative models as well as the privacy security problem in face gen-eration tasks.Firstly,to alleviate the blurriness problem in VAEs,we propose to train the Auto-Encoder branch of VAEs in an unregularized way.We remove the KL(Kullback-Leibler Divergence)constraint on the latent space of VAEs,and train the encoder and the de-coder with the traditional reconstruction error criterion.Thus the latent vectors of train-ing samples will be encoded in a sparse way.In order to sample from the latent space,we also propose a multi-stage GAN,through which the random vectors will be mapped into the encoded latent space.Besides,we also match the decoded distribution of training images with that from random vectors.The experimental results show that our model effectively alleviate the blurriness problem of VAEs,and could generate samples of better quality.Secondly,to tackle the unstable training problem in GANs,we propose to incor-porate self-supervised method into the adversarial training process of GANs.We in-corporate an auxiliary projective transformation estimation objective into the training process of GANs.During the training process,the discriminator tries to estimate the transformation based on both of original images and their transformed counterparts,while following the adversarial training scheme of the original GANs.We propose the intermediate feature matching methods and the feature-transformation matching method to transfer knowledge from the discriminator to the generator.The experimental results show that the proposed unsupervised model reduces the dependence of the discrimina-tor’s representations on the quality of the generator’s output,and drastically reduces the gap between unsupervised and supervised GANs.Thirdly,to tackle the privacy security problem in image generation tasks,we pro-pose an identity transformation GANs framework.First,we explicitly enforce the disen-tanglement between the identity and the attributes of input images.Then,we find proper paths in identity space to transform extracted identity vector.We design a controllable identity transformer through which we are able to control the level of the identity re-moval and produce diverse results with no need for reference images.Extensive ex-periments show that our model can produce results more realistic with better attributes(e.g.,pose,expression,background)preservation.Also,the proposed framework can be easily applied in both image and video streams. |
