Brain Memory Mechanism-inspired Remote Sensing Images Understanding Memory Model

The tasks and data are often dynamic and incremental in dynamic business scenarios such as high-frequency observations and mission bursts,requiring image understanding models that can respond quickly to new tasks and image data.However,due to the dynamic openness of the real world,new task data often deviates from historical observation data,requiring manual intervention to achieve rapid model adjustment and update.Traditional processing paradigms often pass the acquired edge terminal data back to the central server and retrain the model with historical data before processing and analyzing,which is time-consuming and consumes large computational resources.In the face of dynamic incremental task data,how to quickly update the central server and quickly process and analyze the edge terminal task data is a challenging issue in the application of remote sensing image intelligence understanding technology.To address this problem,we argue that the root cause is that the current remote sensing image understanding model based on the closed world assumption is a static learning paradigm,which cannot meet the demand of the real open world dynamic learning paradigm.Around this problem,comparing the brain adaptation to the open world mechanism,we believe that its core lies in the lack of human-like memory capability of the current model,especially the ability to remember across task scales and to continuously learn to update task knowledge for rapid iterative evolution,i.e.,lifelong learning.In this paper,we aim to build a memory model for remote sensing image understanding with cross-scale memory by drawing on the brain memory mechanism to achieve lifelong learning.It has three basic features: reusing historical task-related knowledge to facilitate rapid learning of new task data(memory recall),incrementally accumulating and maintaining historical tasks with low computational resources and data storage(memory retention),and accumulating as much task knowledge as possible with a limited parameter scale(memory adaptation).(1)Aiming at the lack of systematic theory of current memory model for remote sensing image understanding,the basic framework and design principles of deep memory model for remote sensing image understanding are proposed by combining the commonality of currently available deep memory models and brain memory related theories.Through the architecture of dual memory of long-short time network,the model on the central server is modeled on the historical data and the model on the edge terminal on-board model on the current added data,respectively,and the memory recall,retention and adaptation functions are realized through the memory association recall mechanism,memory consolidation transfer mechanism and memory active forgetting mechanism,respectively.(2)In response to the problem of anterograde forgetting in memory recall in the framework of deep memory models for remote image understanding-remembered historical tasks impair current task learning when they conflict with currently learned task representations-the root cause is analyzed to be the inherent problem of inflexible memory capacity in single-memory model architectures,importantly due to the lack of an effective memory recall mechanism.An asymmetric recurrent memory network with decoupled long-and short-term memory is proposed by combining the theory of dual memory model and memory association mechanism of the brain.The long-and short-term memory networks are used to iteratively learn to integrate new task knowledge(short-term memory)and migrate to consolidate historical task knowledge(long-term memory).On this basis,based on the dual-network differential learning mechanism,the unique characteristics of high-resolution remote sensing image representation-scale dependence and spectral channel feature dependence-are purposefully considered and analyzed,and a synergistic asymmetric structure is introduced to effectively improve the ability to learn new tasks quickly on remote sensing image understanding tasks.The experimental results show that the proposed algorithm has a significant advantage over the classical algorithm in the parsimonious forgetting index and outperforms similar algorithms by nearly 10% in the corresponding index of lifetime learning for remote sensing image scene classification tasks.(3)Aiming at the problem of catastrophic forgetting in memory retention in the framework of deep memory models for remote image understanding-learning new tasks leads to severe degradation of the model’s performance on historical tasks,the current synaptic memory consolidation theory is focused on analyzing that the reason for the intensification of catastrophic forgetting when learning and remembering a large number of tasks lies in the cumulative effect of generalized decompression space contraction.On this basis,the mechanism of long-range suppression(LTD)is introduced and a method based on synaptic consolidation and neural pruning is proposed to overcome long-range catastrophic forgetting.The experimental results show that the proposed algorithm significantly outperforms similar classical algorithms in overcoming catastrophic forgetting indexes under the same parameter size conditions.(4)To address the problem of memory generalization breakdown in memory adaptation in the framework of deep memory models for remote image understanding-the model fails to learn and remember when the number of learned tasks exceeds the upper limit of the model’s capacity.With reference to the brain’s active forgetting mechanism,selective forgetting in the memory encoding and consolidation stages was found to be the key to solving this problem.Based on this,a deep neural network learning algorithm based on the active forgetting mechanism is proposed by mimicking the active forgetting mechanism of brain inhibition in concert with lateral inhibition.This paper revisits the problem that current research on intelligent understanding of remote sensing imagery dominated by deep models has difficulty in quickly processing learning new task data and updating models under the central processing and analysis-edge device observation model,and reveals that the root cause of this problem lies in the lack of crosstask memory of deep models with the basic assumption of closed world capability,which prevents rapid and continuous learning and knowledge expansion(lifelong learning).The current deep memory models and the corresponding biological theoretical basis are reviewed and summarized from the taxonomic perspective of memory,and a basic framework of deep memory models and corresponding design guidelines are proposed.Based on this framework,a concrete implementation is made in a remote image understanding scenario for lifelong learning,and the problems of existing memory consolidation and forgetting mechanisms in this framework are addressed,and its effectiveness is verified on a series of experiments.There are 62 figures,7 tables,and 230 citations in this thesis.