Research On Noise Reduction Method Of Relation Extraction Based On Distant Supervised

In today's era of big data,data is expanding at an exponential rate.How to quickly extract effective information from massive data has become more and more important.As the core task of Information Extraction,The major idea of Relationship Extraction is to get the semantic relationship between entity pairs from unstructured text.In recent years,the Relation Extraction plays an important role in the fields of Text Summarization,Knowledge Graph,Knowledge base Q ? A and Machine Translation and It is a hot research topic in the field of NLP.The Relation Extraction method based on Deep Learning has been widely used in recent years,including the Supervised Learning Relation Extraction and the Weak Supervised Learning Relation Extraction.Due to the higher cost and lower efficiency of Supervised Relation Extraction,the Distant Supervised Relation Extraction had been proposed.The Distant Supervised Relation Extraction is a kind of Weak Supervised Learning method and its training corpus is constructed by aligning the unlabeled corpus with the Knowledge Base.However,due to the strong assumption of Distant Supervised,there are noise labels(incorrectly labeled labels)in the training corpus which influence the effect of Relation Extraction.In addition,in the Distant Supervised Relation Extraction task,there are some words unrelated to relationship types in most of the sentences,namely noise words,which also have a certain negative impact on Relationship Extraction.Therefore,aiming at the two kinds of noise problems,this paper proposes PCNN + AMW model.The main contributions are as follows:(1)Aiming at the problem of inner sentence noise words,this paper uses the potential contact between relationship and entity pairs,approximates the difference between the tail entity vector and the head entity vector to replace the relationship vector,and then calculates the correlation between the approximate relationship vector and each word at the input layer in order to assigns the higher weights for the words with greater relevance.This method applies the word-level Attention Mechanism,which can reduce the contribution of noise words within the sentence for the relation prediction task and improve the performance of Relation Extraction.(2)Since the traditional convolutional neural network only models the local dependencies of the input information,the Multi-head Self Attention Mechanism can not only capture different semantic information in multiple different semantic representation sub-spaces,but also improve the robustness of local feature dependencies and the robustness of effective features,making the model more explainable and learning semantically richer information.So,aiming at the problem of inner sentence noise words,inspired by the structure of the Transformer model,this paper introduces a Multi-head Attention layer into the PCNN network,that is,the Multi-head Attention Mechanism is used to process the local features of the output of the convolution layer in order to reduce the impact of inner sentence noise words on model semantic understanding.(3)Aiming at the problem of noisy labels,this paper designs a weight loss function related to sample density according to the distribution of cross-entropy loss values.We considers that samples with large loss value and small distribution density may be noise samples.Therefore,this paper sets the loss value of such samples to 0so as to avoid the model spending too much time fitting these samples and reduce the impact of noise samples on model learning.For other samples,the weight of the loss function is related to the loss value density of these samples.Samples with large loss value density and small loss weights are more likely to attract the attention of the model which improve the robustness of the model.To verify the positive impact of the proposed method on Relation Extraction.In this paper,the ablation experiment was carried out on the NYT data set,and the P-R curve,the AUC value of P-R curve and the P@N as a performance index to evaluate models.Finally,we perform comparative experiments on PCNN+ONE,PCNN+ATT,APCCNs,and APCNNs+D models and the result of this experiment indicate that compared with the other models based on PCNN network encoder,the AUC value and the P@N scores of PCNN + AMW model are significantly higher than other models,confirming the effectiveness of our method.