The Research Of Temporal Knowledge Graph Reasoning Based On Reinforcement Learning

With the development of the Internet,high technologies and related industries such as big data,artificial intelligence,the Internet of Things and blockchain,have been deeply integrated and developed,leading to further industrial upgrading.Among them,extracting necessary information from massive data for application is a challenging task.The knowledge graph manages,understands,and utilizes massive amounts of information on the Internet by modeling,storing,and visualizing entities,attributes,and relationships.However,the growing amount of data often exhibits complex time dynamics,and temporal knowledge graphs have emerged to reflect the timeliness of facts.In addition,the sparsity and incompleteness of temporal knowledge graphs induce that many downstream applications require indirect inference rather than direct access to the available data.Temporal knowledge graph reasoning for dynamic events becomes the key focus in this field.This paper conducts research in the following three aspects of temporal knowledge graph reasoning.(1)A method of modeling unseen entities from a semantic evidence view in temporal knowledge graphs is proposed.Emerging entities emerge in the knowledge graphs over time.This method proposes a data preprocessing module based on graph neural networks for emerging entities to obtain their feature expression,which assigns a more reasonable initial embedding for the entity,and enables reinforcement learning to acquire prior semantic knowledge during the process of reasoning.Experimental results show that this method can perform better from a semantic evidence perspective.(2)A method of modeling dynamic entities based on reinforcement learning in temporal knowledge graph reasoning is applied.The underlying characteristics of an entity change over time.This method models dynamic entities based on graph neural networks and self-attention mechanism,associating the entity representation with the dynamic evolution of the entity,providing better potential semantic representation for entities used in reinforcement learning reasoning.Experimental results show that this method can better model dynamic entities and acquire better reasoning results.(3)A method of temporal knowledge graph reasoning by introducing noise into the parameter space.In order to solve the balance problem of exploration and utilization in reinforcement learning,this method improves the exploration ability of the agent by adding Gaussian noise to the strategy network,and dynamically adjusts the noise size through the adaptive noise method,making the agent more likely to discover new and more optimal strategies.The experimental results show that this method can interfere with the reasoning process of the agent to a certain extent and achieve slightly better results.