Relation Extraction Based On Multi-layered Attention Mechanism And Bias Adjustment

With the rapid development of the Internet and mobile Internet in the past ten years,the information on the Internet has exponentially increased,including text,video,audio and other data formats.At this time,we can no longer rely on human resources to obtain and understand this.Huge amount of data.Relation extraction is favored as a basic task of natural language generation and natural language understanding.In the task of relation extraction,the method based on distant supervision is the more popular one.This method uses the knowledge graph to align with the corpus to obtain large-scale labeled data for model training,which not only greatly reduces the dependence on humans in the process of entity relation extraction,but also increases the size of the training set.However,while distant supervision brings convenience,it also adds noisy data to the training set,which has also become the main bottleneck hindering the improvement of distant supervision relation extraction performance.Many studies are devoted to identifying effective examples from these noisy data,thereby improving the accuracy of relation extraction.But most of the methods just deal with each relation independently,ignoring the rich semantic information between the relations brought by the hierarchical relation type system,and this information is crucial to solving the long-tail problem.In response to this problem,the multi-layered attention mechanism proposed in this article,the multi-layered attention mechanism calculates the attention of each example in the bag to different relation layers,so as to obtain a bag representation from coarse to fine granularity.In addition,for the current public distant supervision data set,there is a problem that the relation type distribution of the training set and the test set is quite different,which is one of the reasons for the decline in the accuracy of distant supervision relation extraction.This paper uses a simple and effective adjustment method-bias adjustment to solve the problem of uneven distribution of the relation between the training set and the test set,and proposes two specific implementation schemes.Finally,this article notices that the side information in the knowledge base improves the relation extraction model,and uses the information of the relation phrase and entity type in the knowledge base in the sentence encoder and bag encoder stages to further improve the relation extraction effect.Therefore,based on the above three points,this paper proposes a relation extraction model based on a multi-layered attention mechanism and bias adjustment,which can solve the above three problems to a certain extent.Finally,experiments prove that the relationship extraction model proposed in this paper based on the multi-layered attention mechanism and bias adjustment is slightly better than the mainstream model in terms of PR curve,AUC(Area Under the Curve),and Hits@K performance indicators.The performance in the tail relation far exceeds other comparison models.