Research On Unsupervised Domain Adaptation For Semantic Segmentation Based On Pixel Affinity

Semantic segmentation is the main task of computer vision,which is crucial to intelligent perception system.Driven by deep neural networks,semantic segmentation has achieved great progress.Fully supervised semantic segmentation methods require large amount of accurate dense annotated data,which is extremely expensive to obtain.To ease this issue,researchers propose to leverage synthetic images to train segmentation models.Synthetic data can be generated automatically,which is easier to be collected and has multiple potential applications.However,the performance of models trained on synthetic data drops drastically when applied on real scene due to domain shift between synthetic and realistic data.In this paper,we focus on developing unsupervised domain adaptation semantic segmentation algorithm to tackle this issue.Most existing approaches attempt to adapt domains based on individual pixel-wise information in image,feature,or output level.Different from recent methods,we consider that the output of semantic segmentation is usually structured and has invariant structures across domains.we propose to exploit such invariance across domains by leveraging cooccurring patterns between pairwise pixels in the output of structured semantic segmentation.Based on this,we develop two affinity space adaptation strategies:We proposed a new loss function for unsupervised domain adaption semantic segmentation.Semantic segmentation separates the whole image into non-overlapping regions,in which the pixels share the same semantic label.We observe that segmentation on target images has weak region consistency.Based on observation above,we propose affinity space cleaning loss to enforce adjacent pixels to have the same label prediction.Compared to previous loss functions,our method can generate more balanced gradient of different classes and improve the segmentation accuracy.We proposed a new adversarial domain adaptation framework.Semantic label space shar es similar spatial layout character and coccuring patterns between neighboring pixels.We introduce the concept of affinity space which is build upon affinity between adjacent pixels.Using adversarial training to align affinity space across domains,we could transfer knowledge of co-occurring patterns from source domain to target domain.Compared to previous methods,we are the first time that the affinity relationship is beneficial for unsupervised domain adaptation in semantic segmentation.The proposed method outperforms some state-of-the-art methods.Extensive experiments demonstrate that the proposed two strategies achieve superior or competitive performance against some state-of-the-art methods on several challenging benchmarks for semantic segmentation across domains.