Research On The Entity Recognition And Linking Of Military Data Based On Multimodal Knowledge Graph

Multi-modal entity linking task plays an important role in a series of knowledge-based scene understanding,such as multi-modal information retrieval and multi-modal knowledge question and answer.The main difficulties of entity identification and linking methods of multi-modal military intelligence lie in the multi-dimension of information modes and the complexity of calculation cost,The complexity of calculation cost is that in the traditional entity linking method,too few candidate entities will lead to lower accuracy of final linking,and too many candidate entities will lead to higher calculation cost,slow operation speed and difficulty in real-time dynamic processing.In order to solve the above problems,this paper proposes a method of military intelligence entity discovery and linking based on multimodal knowledge graph.In the discovery of multimodal military intelligence entities,the multimodal context representation is encoded by the discovery method based on multimodal interaction module,and the discovery of multimodal entities is realized by the sequence encoder.· A military intelligence entity discovery model based on multimodal knowledge graph is proposed,which is used to discover multimodal entity references from multimodal military intelligence.According to the visual information and text information of multimodal military intelligence,the model extracts monomodal feature representations respectively,and inputs them into the multimodal interaction module to fuse different modal features.· A military intelligence entity linking model based on multimodal knowledge graph is proposed,which is used to link multimodal entities in multimodal military intelligence.The model uses multimodal pre-training language model to realize multimodal feature coding of context and candidate entities,realizes multimodal feature interaction through multimodal attention mechanism.· Experiment verification: Experiments show that the military intelligence entity identification and linking method based on multimodal knowledge graph proposed in this paper is superior to other multimodal entity identification and linking methods.The research of entity recognition and entity linking technology for multimodal data in military field can solve the increasingly rich problems of multimodal information extraction and fusion,and can better deal with the tasks of multimodal entity recognition and entity linking in real scenes,and further enhance the practical value,which is of great significance to the development of multimodal field and knowledge graph field.