Research On Knowledge Graph Reasoning Based On Deep Reinforcement Learning

Knowledge graphs have attracted more and more attention in recent years,and have been widely used in intelligent search,question answering and recommendation systems.As one of the key research directions,knowledge graph reasoning is inferred based on the existing information in the current knowledge graph.It can not only infer facts that do not exist in the knowledge graph,but also judge the correctness of existing facts.It has great research significance and application value.Among the current knowledge graph inference models,models based on translation representations cannot solve the problem of semantic diversity,and models based on random walks are more expensive in time and space.This paper proposes a multiple reward structure reasoning model based on deep reinforcement learning.Based on the features of knowledge graph reasoning that are mainly related to sequence decision-making,the inference problem is transformed into reinforcement learning problems.The knowledge graph is regarded as the external environment.The reasoning process is modeled as a Markov decision process;the problem query behavior is regarded as an agent,and the action selection is guided by a strategy network.The strategic network mainly includes a long-term and short-term memory network layer and a multi-headed self-attention network layer,which encodes the historical state of the query process and guides the next action selection.The reward function in the model is set to a multivariate structure,and different reward values are given according to the different relationships between the current entity and the target entity found when the agent walks upstream in the knowledge graph to a stop state,and finally calculates the cumulative reward.The objective function of the model is to maximize the cumulative reward,and the training method uses the strategy gradient.The paper conducts knowledge graph reasoning experiments such as link prediction and fact prediction on the NELL-995,FB15K-237 and WN18RR data sets,and compares it with translation representation-based methods and random walk-based methods.The results show the effectiveness of the proposed method better.