Research On Distant Supervision Relation Extraction Algorithms Based On Attention Mechanism

Relation extraction is a common task in the field of natural language processing,which can extract the relationship expressed between entity pairs from a large amount of text,which is used for downstream tasks such as knowledge graph completion and question answering systems.Among them,distant supervision relation extraction method that can automatically generate labeled data has gradually become a research hotspot.The existing distant supervision relation extraction models mainly have the following deficiencies: First,many models in the remote supervised relationship extraction task do not perform comprehensive feature extraction on textual context information;second,many models in the distant supervision relation extraction task do not effectively fuse local and global features;third,most models ignore The hierarchical information of the relationship itself and the potential connection between the relationship and the relationship have not been deeply explored;finally,the long-tailed distribution problem of the data set has not been paid enough attention,resulting in the deviation between the trained model and the real scene performance.In response to the above problems,this thesis has carried out in-depth research work,and the main contributions are as follows:1.Investigate scientific research literature in related fields such as natural language processing,relation extraction,distant supervision,etc.,conduct in-depth research on existing remote supervision relation extraction algorithms,and analyze and summarize the shortcomings and improvements of existing models.2.A sentence-level distant supervision relation extraction model(SS-Att)based on self-attention-gated fusion units is proposed.The existing sentence-level distant supervision relation extraction models have the problems of insufficient context feature extraction and insufficient fusion of local features and global features.Therefore,the model in this thesis incorporates new text features to enhance the representation of the input features,and uses the gating unit to combine the self-attention module with the piecewise pooling convolutional neural network structure.While extracting more comprehensive contextual features,local The features are effectively fused with the global features to obtain deeper features,thereby improving the effect of the model in the distant supervision relation extraction task.3.Based on the SS-Att model and the hierarchical relation modeling algorithm,a new self-attention-based multi-label distant supervision relation extraction model(MLANFM)is proposed.Most of the existing distant supervision relation extraction models are based on single-label classification,while distant supervision relation extraction is actually a multi-label classification task.Based on the adaptation and improvement of the proposed model,this thesis proposes and designs a hierarchical relation modeling algorithm for in-depth mining of hierarchical relation feature information and relation enhancement of text sentence features.At the same time,a new loss function is designed to adaptively adjust its weight update during gradient descent according to the distribution of data categories,so as to alleviate the impact of uneven data distribution and improve the accuracy of multi-label classification.4.For the above two models,compare and analyze them with a variety of advanced distant supervision relation extraction models on the open standard data set,and achieve better results in the corresponding indicators.The experimental results verify the effectiveness of the two models proposed in this thesis.