Algorithms And Applications Of Image Translation Via Deep Learning

Image translation is an important branch of computer vision and computer graphics.It converts one or more attributes of images into another or more attributes,such as dark images captured at night into bright images captured during the day,winter landscapes into summer landscapes,damaged old photos into good old photos,and more.Through image translation,applications such as image editing,special effects generation,old photo restoration,domain adaptation,etc.can be realized.In recent years,image translation based on generative adversarial networks has made great progress.However,this technology still has problems such as low quality of complex image translation,weak translation controllability,and serious dependence on paired data.In order to improve the practical application ability of image translation,this thesis conducts research on the aspects of image translation quality,image translation controllability,few-pair image translation,and image translation application—domain adaptation.The main innovations and contributions of the thesis are as follows:In terms of image translation quality and image translation controllability,the thesis proposes to use exemplar images to provide guidance for image translation.By learning the cross-domain correspondence,the thesis enables the network to understand the semantics of different objects in the image,and generate objects with corresponding styles in the exemplar image.In order to learn cross-domain correspondences without annotations,the thesis proposes to train them together with image translation to learn cross-domain correspondences in a weakly supervised manner.With learned crossdomain correspondence,the thesis proposes a general exemplar-based image translation method.This is the first method that can use exemplar images to finely control the style of generated images at the instance level.At the same time,the results show that the correspondence and exemplar-based image translation can greatly improve the controllability and quality of image translation,and can be applied to a large number of tasks,such as image editing,makeup transfer,and so on.In terms of image translation with few-pair samples,the problem of improving the quality of supervised image translation is studied in the thesis,especially in the condition of limited paired data,and explores the use of a large amount of unsupervised data to help the training of supervised paired data.An image translation method based on latent space mapping is proposed.Experiments show that the efficiency of latent space mapping for paired data is higher than that of image space mapping.In the case of limited supervised data(e.g.20%),the improvement of latent space mapping is more obvious.In addition to the general latent space mapping framework,the thesis also designs a triplet-domain translation method and a Partial Nonlocal block for restoring structural degradation for the real old photo.Combined with the local branch,the proposed method can restore real old photos with multiple degradations.In terms of image translation application—domain adaptation,a domain-adaptive segmentation method based on an image translation mechanism is proposed.The thesis utilizes pseudo labels to assist image translation,and proposes a prototypical pseudo label denoising algorithm,which rectifies incorrect pseudo labels in real-time by using the relative distance information between features and prototypes.Further,the thesis learns a compact target domain feature space through strong and weak data augmentation to assist pseudo label denoising,further improving the performance of the unsupervised domain adaptive segmentation.Experiments show that the proposed method generalizes from GTA5[1]and SYNTHIA[2]to Cityscapes[3],the segmentation metrics mIoU reach 57.5 and 55.5,respectively.The adaptive gains are improved by 52.6%and 58.5%respectively over the prior leading approach.