Cross-Domain Remote Sensing Image Semantic Segmentation Based On Generative Adversarial Networks

With the construction and improvement of global high-resolution earth observation systems,obtaining remote sensing images has become much easier.Thus,the amount of data has grown exponentially.This provides data support for the intelligent interpretation of remote sensing images.As one of the fundamental tasks of intelligent interpretation of remote sensing images,semantic segmentation aims to classify each pixel in one image and assign a pre-defined semantic category label to describe the contour information of the target object.The semantic segmentation results are widely used in thematic mapping,intelligent transportation,environmental monitoring,urban planning,and other fields.With the booming development of artificial intelligence technology,deep learning methods have gradually become the mainstream method of remote sensing image semantic segmentation,and have achieved comprehensive surpassing of traditional image segmentation methods.However,the superior performance of deep semantic segmentation networks highly depends on massive labeled target data,which will cost a lot of time and money.A common method to reduce the dependence on annotated data is to use existing labeled datasets(source domain)for training and testing or inference on the unannotated dataset(target domain).However,due to significant differences between different remote sensing image datasets,there are serious phenomena of the same object with different spectra and different objects with the same spectra.Traditional deep learning models require similar data distribution between source and target domains,which will inevitably lead to difficulty in generalizing the semantic segmentation model trained in the source domain image scene to the target domain image scene.In response to this problem,this paper investigates the problem of cross-domain semantic segmentation of remote sensing images,proposes a series of innovative solutions,and verifies the effectiveness of the proposed solutions through experiments.The main work and contributions of the paper are as follows:(1)A domain adaptative semantic segmentation method based dual generative adversarial network is proposed.First,a generative adversarial network(GAN)is trained to map source domain images to the style of the target domain,reducing the pixel-level distribution differences between source and target domain images.In order to better transfer the network trained on a source domain dataset with rich labels to an unlabeled target domain dataset,multiple weakly-supervised constraints of joint translated image semantic segmentation loss,target domain image pseudo-label loss,and target domain image rotation consistency loss are innovatively introduced.To fully utilize the role of multiple constraints and avoid network degradation,a dynamic optimization strategy is proposed to dynamically adjust the weight of the constraint terms in the objective function during training.(2)A domain adaptive semantic segmentation method based on semanticpreserved generative adversarial network is proposed.The mainstream methods directly use existing generative adversarial network models for source-to-target domain image translation.However,the existing generative adversarial network-based image transfer models do not fully consider semantic information,which often leads to deviation in the translated images and limited transfer performance.To address this issue,representation-invariant constraints and semantic-preserved constraints are introduced into the generative adversarial network model to achieve semantic-preserved sourceto-target image translation.In addition,considering the difference in category distribution between the source and target domain datasets,a class distribution alignment semantic segmentation module is proposed.The class distribution alignment semantic segmentation module includes two levels: At the input level of the model,the class mixing operation(Class Mix)is introduced to randomly select certain classes from the translated image by random sampling strategy,and paste the corresponding image patches onto the target domain image to form a mixed image.The translated images with the mixed images are fed into the semantic segmentation network for training.At the model output level,boundary enhancement constraint is proposed to refine the performance of the target object boundary.Cross-domain remote sensing image segmentation is a challenging emerging topic with strong practical value in the era of remote sensing big data.In this paper,a series of related researches are carried out with generative adversarial network image translation as the main line.Firstly,existing GAN model is used for image translation from the source domain to the target domain,reducing the difference in pixel-level distribution between the two domains.Second,unsupervised information of the target domain data,such as pseudo-label loss and rotation invariant loss,is fully exploited to further improve the performance of semantic segmentation models in the target domain.Subsequently,in response to problem of the bias in the translated images by existing GAN,a semantic-preserved GAN is proposed.Finally,the issue of differences in class distribution between the source and target domain data is also addressed by introducing a class distribution alignment semantic segmentation module,which aligns the distributions at the input and output levels.Numerous experimental results demonstrate that the proposed methods achieve optimal results in classic cross-domain remote sensing image semantic segmentation tasks,proving the effectiveness and generality of the proposed method.This provides a new paradigm for cross-domain remote sensing image semantic segmentation tasks.