| In recent years,benefiting from a series of revolutionary techniques based on deep learning theory,intelligent understanding of high-resolution remote sensing imagery has made impressive progress and played a significant role in application areas such as land management,forest research,and military surveillance.However,due to the deep convolutional neural network(DCNN)end-to-end automatic feature learning paradigm,these high-resolution remote sensing imagery applications often can only obtain the final prediction results of the model.Still,they cannot explain the reasons affecting the model’s generalizability in the decision-making process.Therefore,it is widely believed that remote sensing image intelligence models are black-box models,leading to the models’ trust problem.On the other hand,due to the adversarial example problem of deep learning,i.e.,adding an imperceptible perturbation attack to the original image can make the model produce completely wrong results.This problem makes the prediction results of these applications severe security risks.These security risks also cause trust problems for intelligent remote sensing images models.Therefore,this study attributes the model black box problem and the adversarial example problem to the trust problem of remote sensing image understanding.These trust problems seriously limit the improvement and development of intelligent models for remote sensing images.Human vision,in contrast,is by far the most trustworthy system that can be referenced.Its trustworthiness stems from the fundamental assumption of neuroscience: human vision learns a low-dimensional manifold embedded in a high-dimensional data space.The lowdimensional manifold has strong invariant representational,i.e.,strong recognition stability for factors such as illumination,Spatio-temporal heterogeneity,and distorted deformation.Inspired by that,the black-box problem of intelligent models of remote sensing images under invariant representations needs to study the relationship between invariant representations and generalizability.It contains the explanatory properties of the invariant representation,i.e.,the relevance to the model’s generalizability.And the model cannot interact with the invariant representation of remote sensing images,i.e.,the constructed invariant representation is used to enhance the model’s generalizability.Similarly,under the invariant representation,the essence of the adversarial example problem is the inability to maintain the stability of the invariant representation of the remote sensing image,i.e.,the perturbation attack destroys the invariant representation of the objects and turns the objects into another class.As well as,the model cannot achieve robustness of the invariant representation,i.e.,defending and recovering from the effects of the perturbation attack.In summary,this study systematically proposes a trustworthy intelligent understanding system for high-resolution remote sensing images: based on the invariant representation of remote sensing images,the four basic properties of Explanatory,Interaction,Stability,and Robustness are systematically analyzed from the perspective of invariant representation.And the correlations among these four basic properties are also described.Under the trustworthy intelligent understanding system of high-resolution remote sensing images,this study further proposes the following algorithms:(1)Generalizability Measure of Remote Sensing Image Scene Classification Model Driven by Activated Representational SubstitutionThe black-box problem of the remote sensing image intelligence model firstly requires the construction of remote sensing image invariant representation and the analysis of its correlation with the model’s generalizability.This study considers the convolutional kernel as the basic unit of DCNN to extract features,which naturally correlates with the model’s generalizability.So,this study proposes an Independent Activation Maximization(IAM)algorithm as a visual invariant representation learned by convolutional kernels.Then,an Activated Representational Substitution(ARS)metric is proposed based on the IAM algorithm.Without a test set,ARS can quantify the importance of representations and measure the generalizability of the whole remote image recognition model.Experiments show that when the model has diverse invariant representations,the model has better generalizability.The correlation between this metric and the accuracy calculated using the test set is 0.901.More,ARS can be used to explain other regularization terms and guide the training of models with better generalization.ARS is also the first algorithm that can quantify the generalizability of a model without using a test set.(2)Self-feedback Optimization Method for Remote Sensing Image Scene Classification Driven by Invariant Visual RepresentationsTo further improve the generalization of the intelligent model for remote sensing understanding under interpretability and to achieve interaction with invariant representations,it is necessary to use invariant representations to improve the model’s generalization.In this regard,this study proposes an adaptive optimization procedural learning method based on the results of ARS metrics to make remote-sensing intelligent models with diverse invariant representations.It addresses what features to transfer from models with diverse representations and how to transfer them.Experiments demonstrate that procedural learning can effectively improve the model generalization and convergence speed with an average accuracy improvement of 7.68%.With this study,the algorithm is also the first training method that uses the internal representation signal from the pretrained model to assist the downstream task.(3)Adversarial Examples Characteristic and Pattern Analysis for Remote Sensing Image Scene ClassificationUnder the trustworthy Intelligence understanding system,the adversarial example problem is due to the perturbation attack that destroys the invariant representations.6 remote sensing images and 8 CNNs are selected to create 48 classification scenes further to analyze the adversarial examples’ characteristics and patterns.These models are then attacked using the 4 most commonly used attack algorithms for 192 attack scenarios.The following conclusions are found:(a)The prevalence of adversarial examples exists,and the fooling rate of all 192 attack scenarios is higher than 98%.(b)The richer the feature information is,the more vulnerable the optical image is,susceptible to perturbation attacks.(c)Adversarial examples of the remote sensing image are highly transferable,and adversarial examples from a single model can attack multiple other remote sensing image intelligent models simultaneously.(d)The distribution of the adversarial examples’ classes is not affected by the model type and attack algorithm,revealing the correlation between remote sensing images.This work is also the first study to comprehensively and systematically analyze remote sensing image adversarial examples to the best of our knowledge.(4)The Adversarial Example Identification Method of Remote Sensing Image Driven by Adversarial Feature GenomeIn order to maintain the robustness of the invariant representation of remote sensing images,the model needs to detect and achieve the correct classification of adversarial examples when they are attacked.This study finds significant differences in feature space between adversarial examples and normal images and thus proposes the Adversarial Feature Genome(AFG).It contains information about the difference between original and adversarial examples and features of the correct class.Further,this study makes detecting and recovering adversarial examples a two-way model problem and proposes an adversarial example identification method for remote sensing images driven by AFG.Experiments show that the adversarial example identification method based on AFG can simultaneously detect and recover the original correct class of adversarial examples with an average accuracy of 85.32%.The method can also maintain accuracy of 79.41% when facing adversarial examples of unknown attacks.As known from this study,the proposed method is also the first study to detect adversarial examples and recover the incorrect recognition results simultaneously.In the light of the above work,inspired by the neuroscience perception of human vision,this study argues that the essence of the trust problems of remote sensing image intelligence understanding is the inadequate understanding of the invariance representation of remote sensing images.And this study proposes a framework and method for trustworthy intelligence understanding of high-resolution remote sensing images.Through theoretical and empirical studies in various remote sensing image intelligence models and remote sensing image datasets,verified the feasibility and effectiveness of the credible intelligence understanding system for high-resolution remote sensing images.60 figures,28 tables,226 references... |
PDF Full Text Request