Deep Transfer And Representation Learning Based Remote Sensing Image Understanding

In remote sensing,synthetic aperture radar(SAR)is one of the most advanced sensor since it has the advantage of acquiring data in day-night and all-weather conditions.With more satellites missions,a large amount of SAR images are available and the automatic interpretation of those is increasingly important.Change detection and land-cover classification are two active research topics in remote sensing image understanding.They have wide application in many disciplines,such as agriculture monitoring,urban planning,disaster evaluation,and environmental investigation.In recent years,deep learning(DL)methods have delivered breakthrough results in numbers of fields,such as image classification,object detection,and semantic segmentation.In remote sensing,there has been a growing interest in the development of DL-based methods to deal with change detection,target recognition,image retrieval,land-cover classification,and so on.On one hand,various supervised deep neural architectures have been exploited to learn efficient high-level representations.They usually require a large amount of manually labeled data.However,it is expensive and time-consuming to label the training data for pixel-level tasks.On the other hand,those generative methods e.g.AE and DBN,are optimized by the pixel-based objectives,e.g.reconstruction loss.The learned representations focus more on the image details,instead of the abstract and contextual information.Moreover,the learned representations lack of correlations between neighboring pixels.In this dissertation,we attempt to address those problems.The researches are summarized as follows,(1)Supervised deep neural networks(DNNs)have been extensively used in diverse tasks.Generally,training such DNNs with superior performance requires a large amount of labeled data.However,it is time-consuming and expensive to manually label the data,especially for tasks in remote sensing,e.g.,change detection.The situation motivates us to resort to the existing related images with labels,from which the concept of change can be adapted to new images.However,the distributions of the related labeled images(source domain)and unlabeled new images(target domain)are similar but not identical.To adapt the concept of change from labeled source domain to unlabeled target domain,we propose a transferred deep learning-based change detection framework.It consists of pretraining and fine-tuning stages.In the pretraining process,we propose two tasks to be learned simultaneously,name1y,change detection for the source domain with labels and reconstruction of the unlabeled target data.The auxiliary task aims to reconstruct the difference image(DI)for the target domain.DI is an effective feature,such that the auxiliary task is of much relevance to change detection.The lower layers are shared between these two tasks in the training process.It mitigates the distribution discrepancy between the source and target domains and makes the concept of change from the source domain adapt to the target domain.To fine-tune the change detection network(CDN)for the target domain,two strategies are exploited to select the pixels that have a high possibility of being correctly classified by an unsupervised approach.The proposed method demonstrates an excellent capacity for adapting the concept of change from the source domain to the target domain.It outperforms the state-of-the-art change detection methods via experimental results on real remote sensing data sets.(2)Supervised change detection methods always face a big challenge that the current scene(target domain)is fully unlabeled.In remote sensing,it is common that we have sufficient labels in another scene(source domain)with a different but related data distribution.To detect changes in the target domain with the help of the prior knowledge learned from multiple source domains,we propose a change detection framework based on selective adversarial adaptation.The adaptation between multisource and target domains is fulfilled by two domain discriminators.First,the first domain discriminator regards each scene as an individual domain and is designed for identifying the domain to which each input sample belongs.According to the output of the first domain discriminator,a subset of important samples is selected from multisource domains to train a deep neural network(DNN)based change detection model.As a result,not only the positive transfer is enhanced but also the negative transfer is alleviated.Second,as for the second domain discriminator,all the selected samples are thought from one domain.Adversarial learning is introduced to align the distributions of the selected source samples and the target ones.Consequently,it further adapts the knowledge of change from the source domain to the target one.At the fine-tuning stage,target samples with reliable labels and the selected source ones are used to jointly fine-tune the change detection model.As the target domain is fully unlabeled,homogeneityand boundary-based strategies are exploited to make the pseudolabels from a preclassification map reliable.The proposed method is evaluated on three SAR and two optical data sets,and the experimental results have demonstrated its effectiveness and superiority.(3)Due to the complementary properties of different types of sensors,change detection between heterogeneous images receives increasing attention from researchers.However,change detection cannot be handled by directly comparing two heterogeneous images since they demonstrate different image appearance and statistics.Inspired by the DNNs designed for semantic segmentation and human pose estimation,we propose a deep pyramid feature learning network(DPFL-Net)for change detection,especially between heterogeneous images.DPFL-Net can learn a series of hierarchical features in an unsupervised fashion,containing both spatial details and multi-scale contextual information.The learned pyramid features from two input images make unchanged pixels matched exactly and changed ones dissimilar,after transformed into the same space for each scale successively.We further propose fusion blocks to aggregate multi-scale difference images(DIs),generating an enhanced DI with strong separability.Based on the enhanced DI,unchanged areas are predicted and used to train DPFL-Net in next iteration.In this paper,pyramid features and unchanged areas are updated alternately,leading to an unsupervised change detection method.In feature transformation process,local consistency is introduced to constrain the learned pyramid features,modeling the correlations between neighboring pixels and reducing the false alarms.Experimental results demonstrate that the proposed approach achieves superior or at least comparable results to existing state-of-the-art change detection methods in both homogeneous and heterogeneous cases.(4)The most generative methods are optimized by pixel-level objectives,e.g.reconstruction loss.The learned representations focus more on the image details,instead of abstract and discriminant information.In order to detect change in both homogeneous and heterogeneous images,we propose a contrastive self-supervised learning based method to learn dense image features.By taking account of the distribution characteristics of speckle noise in SAR images,the novel contrastive self-supervised learning method can extract dense features.In addition to data augmentation,the correlation between pixels is used in building positive pairs to design a better pretext task for dense prediction.Based on the learned dense features,we propose an unsupervised change detection method where the paired pixels in unchanged area can be matched exactly.The proposed change detection method achieves superior results in both homogeneous and heterogeneous cases.(5)Though supervised deep learning has achieved great success in extracting hierarchical features for various image applications,it usually requires a large amount of manually labeled data.Generally,it is expensive and time-consuming to label the data,especially for pixel-level tasks.We propose a coarse-to-fine contrastive self-supervised learning framework to extract global and local features for land-cover classification with limited labeled data.It consists of two stages,one for encoder pre-training to learn global features and the other for decoder pre-training to derive local features.Firstly,a novel contrasting strategy is introduced considering the geometric position and semantic meaning of different regions.It provides the self-supervised methods more cues to learn than those only relying on data augmentation.With the novel contrasting strategy,global and local features are learned by forcing the positive pairs defined by nearby regions with similar semantic meaning close to each other and negative pairs of those dissimilar apart.Secondly,a discriminant constraint is incorporated into a classification model with an encoder-decoder architecture to maximize the inter-class distance.It is more competent to distinguish between different categories that have similar appearance.Finally,the classification model is validated on three SAR images with limited labeled data for land-cover classification.After pre-training,the classification model with the discriminant constraint improves the experimental results substantially.It also demonstrates the effectiveness of the proposed method in pixel-level tasks after comparison with the state-of-the-art methods.