Remote Sensing Scene Interpretation Based On Feature Representation And Hash Learning

As an essential task of remote sensing(RS)image interpretation,RS scene interpretation is a systematic project,which has great potential for military and civilian applications.In recent years,with the development of sensor technologies,the volume of high-resolution RS(HRRS)images has increased explosively,which poses a serious challenge to the RS scene interpretation.On the one hand,the contents within HRRS images are complex.The objects within one HRRS scene are diverse in type,huge in volume,and various in scale,which increases the difficulty of HRRS scene interpretation.Therefore,how to extract the scene-level represen-tations from the complex HRRS images is an urgent problem.On the other hand,in the era of big data of HRRS images,managing these large-scale RS images efficiently and retrieving the specific contents accurately from such large archives in accordance with users’ require-ments becomes a necessary task for scholars.This thesis proposes a series of methods with the consideration of the two mentioned issues to accomplish HRRS image scene interpreta-tion.These methods are grouped into two sets,i.e.,scene classification and content-based image retrieval.In detail,1.A multi-scale feature learning based on multi-branch networks for the HRRS image scene classification method is proposed.In this method,the original HRRS images and their down-sampled versions are fed into the multi-branch networks to capture the local and global fea-tures,which could help overcome the influence of the objects with varying types and resolu-tions in HRRS scenes.The local features are obtained by integrating different convolutional features.Some fully connected layers are then selected for learning local feature mapping matrix from integrated features to extract the discriminative features from the local features.The global features are used to exploit the global structure mapping matrix to encode and in-tegrate the discriminative features into RS scenes’ features,improving the extracted features in RS image scene classification tasks.2.Through changing dilated rates,the dilated convolution has different reception fields.Based on this characteristic,a self-adaptive pyramid convolution block is developed,and we apply it to RS image scene classification.This pluggable convolution block can be em-bedded in any existing CNNs for capturing multi-scale features from HRRS images.The self-adaptive pyramid convolution contains two components,i.e.,pyramid convolution and scale self-adaptive scheme.Specifically,the pyramid convolution utilizes the same convo-lutional kernel to complete the dilated convolution with different dilated rates for captur-ing the multi-scale features,enhancing the ability of features learning for RS images.The scale self-adaptive scheme is developed for fully utilizing the complementary information between different scales features where correlation coefficients of different scales features are calculated by analyzing the information inherent within these features.These features could be fused by the coefficients and improved their representational ability.Based on the fused multi-scale features,the performance of the RS image scene classification task could be improved.3.A supervised deep hash learning method based on generative adversarial networks is pro-posed.In this method,a feature learning model is introduced to improve the ability of fea-ture learning for HRRS images.For the feature representation,a feature aggregation scheme is developed to capture the multi-scale features.An attention mechanism is designed to strengthen the essential local features that are discriminative for HRRS images’ scene cate-gory.To improve retrieval efficiency,an adversarial hash learning model is proposed that maps the obtained features into compact binary hash codes.A specific loss function is de-signed to satisfy the similarity preservation and minimize quantization errors.Besides,a generative adversarial learning framework is used in our model for posing a binary discrete uniform distribution constraint on the learned hash codes.Such that,the learned hash codes could be coding balanced and further to reduce the quantization errors.4.A semi-supervised deep hash learning method based on adversarial auto-encoder is pro-posed.First,a convolutional neural network is used to learn the high-dimensional dense features from HRRS images.Then an auto-encoder model is constructed for encoding and decoding these dense features.Specifically,the hidden layers’ features of the auto-encoder are divided into category variables and hash codes.The auto-encoder model could be opti-mized by the labeled/unlabeled samples and improve the two hidden variables’ representa-tional ability.Second,to make the categorical variables and hash codes close to the optimal results as possible,a category distribution discriminator and a uniform distribution discrimi-nator are proposed to pose some suitable distribution constraints to the categorical variables and hash codes improve the representation ability of the hidden variables.Finally,for the labeled samples in the auto-encoder training process,a cross-entropy loss function is used to enhance the categorical variables’ confidence.A triplet loss function is used to maintain the hash’s validity coding.Meanwhile,for the unlabeled samples,the labeled and unlabeled samples’ high-dimensional dense features are used to calculate an adjacency matrix to obtain the auxiliary information.Also,other auxiliary information could be obtained by the cate-gorical variables.These two kinds of auxiliary information jointly assist the hash learning of unlabeled samples and enhance the generalization performance of the hash learning model.5.A deep hash learning algorithm based on meta-learning is proposed.The method uses meta-learning to solve the hash learning problem.It improves the performance of hash learn-ing with few training samples.This algorithm develops different distance measurements to explore the similarities between support and query sets and the similarities between sam-ples within support sets.Based on these distances,a specific hash learning loss function is designed for overcoming the interference of intraclass diversity and interclass similarity.Therefore,it can improve the performance of hash learning for HRRS image retrieval task.To further enhance the proposed method,a dynamical version is developed,adjusting the number of categories and samples within the support set and query set.The dynamic ver-sion could further improve the generalization of the hash learning model under few training samples by introducing various intraclass diversity and interclass similarity.