| With the development of remote sensing technology,the acquisition of remote sensing data is more convenient.Remote sensing data reflects the natural and social activities on the earth’s surface.At the same time,massive data also challenges how to transform data advantage into information advantage.At present,the research on remote sensing image mainly focuses on classification,detection and segmentation,but these tasks are not enough for the mining and understanding of remote sensing image information.Remote sensing image captioning task can intelligently extract and describe the main content of remote sensing image,including the recognition of the category,quantity,color and other attributes of objects in the image,as well as the reasoning of the relationship between the scene and the object and between one object and others,and finally use the natural language that human can directly understand to describe the image content.By generating concise and accurate sentences to summarize and present the content of remote sensing image,it is convenient to understand the content of image.It is of great research significance and use value to study the task of remote sensing image captioning.However,due to the complexity of remote sensing image content,it is a challenge to generate appropriate description.In order to build an accurate content description model of remote sensing image,this thesis works from the following aspects:(1)From the perspective of using the high-level attribute information of the image,a content description generation model of remote sensing image based on attribute attention mechanism is proposed.Using the encoder decoder framework,the global attributes of remote sensing images are introduced into the model and combined with attention mechanism,so as to further improve the sensitivity of the remote sensing image scene visual information and improve the description ability of the model for multi-objective scene.The experiment shows that the model has a good effect on the remote sensing images with complex scene and diverse objects.(2)From the perspective of making the distribution of attention weight more reasonable and effective,a content description generation model of remote sensing image based on multisource interactive hierarchical attention mechanism is proposed.The channel semantic information of remote sensing image is extracted by channel attention mechanism,and the multi-source interactive attention mechanism is designed to make better use of image and text information,and then the human visual mechanism and cone cell distribution rules are used for reference to design a hierarchical attention reallocation model.Combined with the double-layer long and short-term memory network,the ability of understanding and reasoning the intermediate state vector is improved.The experimental results show that the model can make full use of multi-source image and text information,redistribute attention in spatial domain,further enhance the attention to important objects,and reduce the interference of irrelevant information.(3)From the perspective of enhancing the relevance between attention results and query,a remote sensing image captioning model based on re-attention and reinforcement learning is proposed.Firstly,the re-attention module is designed to constrain and refine the attention results,filter the attention results irrelevant to query,and embed the module into the encoder to further improve the efficiency of image features.By combining the module with the decoder,the state information in the decoding process can be used.In addition,reinforcement learning is added to train the model,which improves the quality of the generated sentences and the ability of describing the objects,attributes and relationships in remote sensing images. |
