Cross-domain Semantic Segmentation Via Adversarial Learning

Semantic segmentation aims to assign a predefined semantic class label to each pixel in an image,enabling a computer to visually understand the scene at a finegrained level.This technology is widely used in tasks such as automatic driving,urban planning and smart home,which is an important branch in the field of computer vision.In recent years,deep convolutional neural network-based segmentation technology has improved the task performance to a new level.However,the existing deep learning methods require a large number of pixel-level manually labeled images as training data.However,this performance promotion is at the price of huge amounts of dense pixellevel annotations obtained by expensive human labor.To alleviate the heavy burden of manually labeled data,an alternative would be resorting to simulated data,such as computer-generated scenes,by which unlimited amounts of labeled data are available.However,due to the different data distributions between different domains,the deep model trained with simulated images cannot be well generalized to the real scene.In view of this problem,this paper makes a thorough study of the domain adaptive problem in semantic segmentation,proposes a series of innovative solutions,and verifies the correctness and effectiveness of the solutions through experiments.The main work and contributions of the paper include the following aspects.(1)We have proposed a macro-micro adversarial network for semantic segmentation tasks.For semantic segmentation problem,the context information is usually difficult to be used.Our method uses a discriminator to guide network segmentation to consider the pixels context and global context information.This strategy makes the model output to be close to the real label,so as to improve segmentation results.The algorithm uses two different discriminators to supervise the segmentation network,which improves the global semantic consistency and local semantic consistency of the segmentation results respectively.Our result is among the state of the arts.(2)We have proposed a category-level domain adaptation for cross-domain semantic segmentation tasks.We analyzed the shortcomings of the existing domain adaptive segmentation methods in the output space and proposed a new domain adaptive segmentation method based on the category-level feature alignment.By combining the idea of co-training and adversarial learning,our method employs two orthogonal classifiers to align the distribution between source and target domain at category-level,which greatly avoids the negative transfer in the domain adaptation process.(3)We have proposed a significance-aware information bottleneck for domain adaptive semantic segmentation.We analysed the drawbacks of the current latent-space domain adaptation methods for segmentation,based on which we have proposed to equip the conventional adversarial network with an information bottleneck.The new network structure enables a feature purification before the adversarial adaptation,which eases the feature alignment and stabilizes the adversarial training course.On multiple benchmarks,our method achieves the state-of-the-art segmentation accuracy and surpasses the domain-adaptive method based on output space.For the semantic segmentation task,our proposed method brings the feature-/output-space UDA methods to the same starting line.(4)We have proposed an one-shot unsupervised domain adaptation framework for cross-domain semantic segmentation task.Compared with using a large number of unlabeled images of the target domain for training,we may face more challenging conditions in reality,for example,only a few or even a single image of the target domain can be obtained.To deal with this challenge,an adaptive segmentation method based on single target domain is proposed.We aim to search for the potential styles in the target domain from a single sample,and then the images in the source domain are transferred to these new styles and are trained in an adversarial manner.Such method can largely increase the generalization ability of the model in the target domain.Cross-domain image segmentation is a new and challenging subject,which has a strong practical value in the era of big data.However,compared with the traditional full-supervised semantic segmentation,there is still a big gap in the segmentation accuracy and segmentation speed.In this paper,only a part of the problems are studied,and the subsequent research will go deep into the neural network principle itself.We aim to find the essential reasons for the performance degradation of the model in crossdomain situations from the perspective of explanatory ability,so as to solve the domain adaptation problem.