Research On Distant Supervision Relation Extraction With Attention Enhanced Bag Representation

As a key link of information extraction technology,relation extraction has important theoretical significance and broad application prospects in the field of automatic construction of knowledge graphs and natural language processing.It provides important support for a variety of applications,mainly in the intelligent question answering system and intelligent search scenarios.Its profound practical value has aroused widespread concern in academia and industry,and numerous theoretical results and application products have emerged.The distant supervision relation extraction technology uses external knowledge base as the supervision source to automatically label corpus data,which saves the cost of manual labeling and has become a research hotspot in the field of relation extraction.Due to the strong assumptions of distant supervision,there is a large amount of noisy data in the annotated corpus.Therefore,the current research focus is mainly on how to weaken the negative effects of noise data.Based on the pre-trained Transformer language model,this paper proposes two distant supervision relation extraction models with attention enhanced bag representation to mitigate the impact of noise as follows:(1)We proposed a relation extraction model DISTRE-EA based on pre-trained Transformer language model and entity attention mechanism.The relation extraction task is intended to identify the relation to be expressed by two entities,so the guiding role of entity representation cannot be ignored.Based on the advantages of the language model,the information of the entity itself is used to guide attention.By calculating the relevant characteristics between the entity representation and each sentence inside the bag,the semantic similarity between the two is deeply explored,and the representation information of the sentence obtained by the pre-training model is screened to reduce the impact of noise sentences and help optimize the bag representation.Relevant experimental data shows that DISTRE-EA is superior to mainstream methods,and effectively validates the extraction effect of the model.(2)We presented a relation extraction model DISTRE-SA based on pre-trained Transformer language model and sentence-level self-attention mechanism.Since the sentence representations within the bag are obtained through pre-trained models,there is a lack of close correlation between each other.In order to break the independent representation of sentences within the bag,this article uses self-attention techniques at the sentence level.Self-attention can not only capture the interdependence of the input sequence,but multi-head attention can also enrich the implicit information of different representation subspaces.After using self-attention to transform the sentence representation,combined with selective attention,the bag representation is further enhanced.The results in the NYT dataset show that DISTRE-SA has higher extraction accuracy than the original method DISTRE.