Face Semantic Translation Using Unsupervised And Semi-supervised Learning Based Generative Adversarial Networks

The essence of image to image translation is to learning the mapping from the source domain to target domain and this mapping may be 1v1,1v many,and many v many.Image to image translation has a very wide application in reality,for example,gray image to color image.Recently,generative adversarial networks has been applied for this field for learning the common architecture,which is for multiple task,e.g.,image style-transfer,image in-painting and domain adaptation field.In details,previous IIT methods can be categorized into two broad classes,including methods that learn from paired training data and methods that learn from grouped training data.These two methods can be considered as the supervised methods.When the quantity of needed groups and training samples for every group is sizeable,it still is labor demanding and requires a certain amount of human resources to complete if no suitable labelled dataset exists in reality.To alleviate these problems,we propose two models,one is sparsely grouped learn-ing method,where only a few of the training dataset is grouped while the remaining unlabelled data is used for unsupervised learning to improve the performance of clas-sification and stabilize the training of the adversarial network.Therefore,this method is actually semi-supervised learning.The IIT tasks with sparsely grouped learning just require every group to have a few but not excessive training samples,where it could reduce the labour for labelled training data and improve the extension of the learning.We name this model as SG-GAN.The other is to learn from mixed dataset,where al-1 dataset is mixed without the labels.This model is unsupervised learning and could utilize the semantics in the mixed training dataset,then producing natural and various transformations between the discovered semantics,just by controlling the latent code.In this paper,we name this method as ST-GAN.To validate the effectiveness of the proposed models,we apply two models for the facial attribute manipulation task.The facial attribute manipulation can be a specific task of image translation on face dataset.For attribute age,it aims to change the appear-ance of the input face from young to old or from old to young and keeps the identity information consistent between the input and output at the same time.Experimental results show that SG-GAN can achieve comparable results with state-of-the-art methods on adequately labelled datasets while attaining a superior image translation quality on sparsely grouped datasets.Moreover,we proposed an adapted residual image learning method to improve the ability of keeping the consistency of irrelevant content.To the best of our knowledge,the proposed ST-GAN is the first work to learn the facial attribute manipulation in the mixed data which is not labelled with any supervised information.Though ST-GAN does not attain high-quality facial attribute manipulation results,but it could capture the semantic information of data by using unsupervised learning and manipulate the special attribute where it is hard to achieve in the previ-ous methods.Moreover,we optimize ST-GAN using local mutual information maxi-mization to make semantic discovery and transformation more explicit and regionally oriented.