Research On Knowledge Reasoning Technology Based On Deep Reinforcement Learning

Knowledge graph describes real-world knowledge networks of concepts,entities,and their relations in the form of structured triples,which is emerging as a paradigm for managing large amounts of information with its excellent scalability and interpretability.Knowledge reasoning plays an important role in the full lifecycle of building stored and applied knowledge graphs,and is known as the crown of knowledge graphs.Among the many knowledge reasoning methods available today,knowledge reasoning based on deep reinforcement learning is undoubtedly one of the emerging and excellent solutions for solving knowledge questions and answers(KGQA).In the context of this background,we focus on knowledge reasoning based on deep reinforcement learning to solve KGQA tasks in knowledge graphs.First,the knowledge graph triples,and the entity and relation of the question are mapped into word vectors by embedding techniques,and the local features of the knowledge graph and the answer to the question are obtained by integrating the word vectors of entities and relations with neural networks and reinforcement learning.Given that,the thesis mainly includes the following:1)In the light of the multi-hop reasoning problem of the knowledge graph,a dynamic knowledge reasoning framework based on deep reinforcement learning is proposed to implement path-based dynamic knowledge reasoning.To address the problem of fixed-step reasoning in existing work,a dynamic reward mechanism is proposed to train a dynamic reasoning model.To tackle the problem that a pre-trained model is needed to construct the reward shaping function during reasoning,the hypothesis of path-based dynamic reasoning is proposed,and a scoring function to calculate the vector similarity of entities and relations is established,which is then employed to implement a reward shaping function.In addition,a non-target reasoning scenario is extended to the existing target-based reasoning case.Besides,two judgment conditions are built for the dynamic reasoning process against two different reasoning situations during the test.Two optimization techniques are also proposed,including optimizing the model parameters with a soft actor-critic policy function and attempting to improve the initialization representation of the dynamic reasoning model using a pretrained model.Finally,a large number of dynamic reasoning experiments demonstrate the effectiveness of the model.2)The concept of valid triples is proposed to address the inherent pattern of building multi-hop question-answer pairs in existing work,and indicate that there are currently a large number of invalid triples in several classical datasets,which can then be removed to create new datasets.To tackle the problem of how to optimize the initialization of multi-hop reasoning models,the pre-trained model for single-step reasoning and the multi-hop reasoning model,as well as the graph attention model are set to use the same word embedding representation.In addition,to solve the problem of how to encode multi-hop paths by graph neural networks,a method is proposed to integrate multi-hop paths of knowledge graphs with graph attention mechanism,weighted encoding of valid and invalid paths of the triples by attention network respectively,employing convolution operation to obtain representation feature scoring of valid paths and invalid paths,and further applying cross-entropy function to calculate the loss values and optimize the parameters.Finally,the enhanced feature representation of the graph attention encoding is fed into a multi-hop inference model for decoding tests,and the experimental results verify the effectiveness and robustness of the model.The experiments are performed on five classical knowledge graphs,and the results show that both model frameworks can improve the performance of existing algorithms in different dimensions.