Research And Implementation Of Knowledge Graph Reasoning Algorithm Based On Reinforcement Learning

Knowledge graphs are widely used.They are not only the upstream tasks of many NLP(Natural Language Processing)subtasks,but also are important parts of recommendation systems and question answering systems.However,there are a large number of missing relations in the structured graphs mainly from various encyclopedias,and there are many error triples in the graphs extracted from Relation Extraction methods.These problems affect the correctness of the graph application system.To solve the above problems,the Knowledge Graph Reasoning(KGR)algorithms can be used to inference the facts in the incomplete graph,and they can also be used to evaluate the quality of the extracted graphs.The contributions of this thesis are as follows:1.Aiming at the problem of wrong relation in the process of entity relation completion,we study the reinforcement learning model based on curriculum learning to solve the problem of false path discovery,that is,the imperfect training sequence(path)obtained by the agent's wandering leads to the agent being misled.The third chapter of this article uses a knowledge-based soft reward method combined with the curriculum learning strategy.This method maps the path obtained by the agent in the environment to the LSTM Policy Network,and then uses entity and relation embedding as a scoring function composed of parameters to constructe soft reward.At the same time,action dropout strategy of the agent is used to randomly discard the outgoing edges on multiple jumps of the path.Finally,a more accurate entity relation path-learning method is realized.The indicators Hit@1,Hit@10,and MRR of MINERVA's improved model increased by1.7%,1.5%,1.3% on WN18 RR,and increased 1.9%,4.8%,3.7% on NELL-995.2.The knowledge graph reasoning methods based on reinforcement learning and the algorithm proposed in Chapter three include the problems of unknown path quality and poor performance.Chapter four of this thesis proposes an improved path quality judgment algorithm in a complex semantic environment to address the false path problem.This method uses additional semantic information to evaluate the quality of the sampled paths by semantic similarity,and then uses the output of the path evaluation module as the reward part of the reinforcement learning process to motivate the agent to choose highquality paths.The indicators Hit@1,Hit@10,and MRR of the improved method increased by 2.2%,6.8%,4.8% on FB15K-237.And the indicators increased 0.6%,2.8%,1.7% on NELL-995.3.This article implements a knowledge graph complement and verification system.As the upstream task of many NLP tasks,the knowledge graph can also be used as a knowledge supplement in the recommendation systems and question answering systems.The knowledge graph completion and verification system designed in this article is used to improve the quality of the graph in the application.We choose to combine the algorithm proposed in Chapter three with the current mainstream front-end and back-end framework to build a system to provide the KG completion,confidence evaluation and querying QA functions.We use the UMLS dataset to test the system and prove the effectiveness and versatility of the algorithm.