Research On Document-level Long Text Relation Extraction Algorithms

With the rapid popularization and development of the Internet,the number of people on the Internet is increasing all over the world,and the magnitude of data generation has also ushered in an explosive growth.How to make full use of massive data to provide people with intelligent information services has always been the goal pursued by computer practitioners.However,the value of information provided by massive unstructured data is limited or inconvenient for downstream application tasks.Therefore,it is necessary to extract structured knowledge from unstructured data.Relation Extraction(RE)is an important task to solve this demand.From the perspective of vertical development,RE is a very basic upstream task in Natural Language Processing(Information Extraction field).As a basic service,it extracts structured knowledge triples from unstructured data for downstream tasks(such as question answer system,knowledge graph,machine reading comprehension,etc.);From the perspective of horizontal development,after more than ten years of extensive research,the RE task itself has developed different task forms according to different scenarios and needs(such as sentence-level RE,joint RE,distant supervised RE,document-level RE,etc.).The long text documentlevel RE mainly studied in this paper is an important and practical research direction which is widely concerned by the research community in recent years.Through the detailed study of document-level RE task,this paper analyzes and expounds two kinds of problems widely existing in the previous research.Firstly,a knowledge enhancement pre-trained model for RE task is proposed to further improve the baseline performance of document-level RE task.Then,aiming at the problem of insufficient ability to capture long-distance dependency information in document-level RE task,a hierarchical graph neural network model is proposed.Finally,based on the previous two works,this paper also explores the low focus but very important research field of noise document-level RE.The main work of this paper includes:1.Document-Level Entity Relation Knowledge Injected Pre-trained Language Model,DERPLM.In this paper,we propose a pre-trained model of entity relation knowledge injection,which is committed to injecting some entity relation knowledge strongly related to the downstream document-level RE task based on the general pre-trained model(PLM),so as to provide a more excellent pre-trained model for the downstream.Specifically,we designed a pseudo label knowledge injection method,which can efficiently inject external knowledge without adding additional parameters,and we designed several novel pre-trained tasks according to the characteristics of document-level RE task,so that external knowledge can be fully integrated into the pre-trained model.2.Hierarchical Aggregation and Inference Network,HAIN.Aiming at the problem that it is difficult to capture the long-distance dependencies between different entity pairs and aggregate effective information with different granularity in document-level RE.This paper presents a hierarchical graph neural network model,which models the document-level RE task as a hierarchical logical reasoning and information aggregation process.Specifically,this paper introduces three different graphs at three different information levels,and proposes a novel hybrid attention mechanism to efficiently aggregate global and local reasoning information for the entities to be classified.3.Denoising for Document-level RE based on Knowledge Distillation,DKD.In this paper,we explore the low attention but very important research direction of noise document-level RE.In order to train a model with relatively excellent performance at low cost in noisy data,based on the previous two works,we realize the transfer of knowledge with the support of knowledge distillation training framework.Specifically,we combine the models in the previous two works as the teacher model,the traditional pre-trained model as the student model,and the noise document-level RE data as the training data.Under the guidance of the teacher model,we can eliminate part of the influence of noise data,so as to train a student model with relatively excellent performance.