Feature Disentanglement Based Deep Learning Methods For Image Re-rendering

Image re-rendering is a challenging and important image processing task.Giving existing image datasets,re-rendering new images which have different features can be used as a data argumentation method to improve the training and testing performance of machine learning algorithms.Re-rendering new images which have continuous poses of a given object can be used for reconstructing corresponding 3D model from a single 2D image.Image re-rendering can also be used in image-to-image transformations.For example,re-rendering a new landscape image which has different illuminations from the original one,re-rendering a new eye-enhanced facial image of the input photo.In recent years,the rapid development of deep learning has shown great advantages in the field of image re-rendering.Among them,since feature disentanglement based deep learning methods are capable of learning interpretable,understandable and controllable image representations,they have become one of the most powerful image re-rendering methods.In this paper,a set of feature disentanglement based deep learning image re-rendering methods are devised and implemented.This work includes two models,namely,a supervised image re-rendering model named Improved TD_GAN(Improved Tag Disentangled Generative Adversarial Networks)with labeled training data,and a weakly supervised image re-rendering model named ADNET(Adversarial Disentanglement Networks)with weakly labled training data.The main work of the thesis as follows:(1)This paper proposed a supervised feature disentanglement based image re-rendering model named Improved TD_GAN.There are two problems with the existing supervised image re-rendering method:First,they can only re-render new images based on discrete feature descriptions,so the eigenvalues of the generated image are limited to discrete values in the training set.Scond,there is a lack of effective indicators to judge the convergence degree,and convergence speed of the model has some room for improvement.In order to solve these two problems,this paper introduces continuous feature tags in TD_GAN to make the model learn continuous feature.And the image generation part introduces WGAN-GP(Wasserstein Generative Adversarial Networks)to improve the performance of the model by using Wasserstein distance to represent the more robust gap between the real distribution and the generated distribution.After experimental verification,the model can re-render the continuous feature and generate the eigenvalues that do not exist in the training set.Meanwhile,the convergence speed is improved to a certain extent compared with the original model,and loss of the discriminative network in WGAN-GP can better reflect the model convergence,the re-rendered image quality of the model also has certain advantages compared with the existing similar methods.(2)A weakly supervised feature disentanglement based image re-rendering model named ADNET is proposed in this paper.This model introduced an object-recognition network and a target feature extraction network to extract the disentangled representations of the input images by adversarial training,re-rendering the specified features of the image in a weak tag dataset scenario.It's adversarial training idea provides a new solution to the disentanglement of image features.After experimental verification,our model demonstrated higher generation performance compared with the existing similar methods.