Research On Noise Reduction Of Relation Extraction Data Based On Reinforcement Learning

With the rapid development and popularization of mobile Internet,people increasingly rely on obtaining information and knowledge from the Internet.Compared with structured data,massive unstructured text data is scattered in every corner of the Internet,which is more common.The supervised relation extraction model can extract valuable entities from sentences,as well as the relationships between entities,which helps to understand the semantic information of sentences.Relation extraction is widely used in the fields of information retrieval,knowledge graph construction and automatic question answering systems.Training the relation extraction model needs a lot of labeled data.In practical applications,it is difficult to obtain large-scale and high-quality annotation data.At present,manual annotation or distant supervision methods are usually used to obtain annotation data.Although manual annotation can ensure the quality of the annotation data,the cost is too high,but the relation extraction data set generated by the distant supervision method will make a large amount of wrong annotation noise data.The existing noise reduction work mainly deals with the problem of wrong labeled entities or the incorrect classification of relations between entities.These methods directly eliminate noisy sentences from the data set,but fail to retain some correctly labeled sentences,which lose a lot of sample information,and do not consider the situation that there are multiple triplets in a sentence in the current large-scale data set.In this paper,we propose a triplet-level noise reduction model TFRE(Triplet Filter for Relation Extraction)based on the reinforcement learning method to achieve relation extraction data set at the triplet level.The model consists of two modules: Triple Filter and Relation Extraction model.The Triplet Filter identifies and removes noisy data,and inputs the retained high-quality samples into the Relation Extraction model.The Relation Extraction model models and predicts the input data,provides feedback values at the triplet level to the filter,and guides the filter to remove noisy data while keeping valid samples as much as possible.The main work and research contents of this paper are as follows.(1)Aiming at the triplet-level noise problem in the distant supervision data set,the Triplet Filter is constructed by using reinforcement learning,the state representation in the model learning process is defined as the feature representation of the entity pairs in the sentence and the triplets.The policy network is constructed and a feedback mechanism is established to distinguish the wrong annotation triplets.The model can achieve triplet-level noise reduction and solve the noise problem from the data source.(2)This paper adopts the Markov Decision Process of the reinforcement learning method,and defines the reinforcement learning modeling elements such as state,action,policy and reward according to the triplet filter task.The goal is to maximize the expectation of the reward value.The parameters of the policy network are optimized by Monte Carlo Policy Gradient algorithm,and realizes the establishment of a noise reduction model through reinforcement learning.(3)At present,the large-scale relation extraction data set has a large number of samples with overlapping relations or nested entities.Therefore,this paper selects two end-to-end relation extraction models DGCNN+ATT and Cas Rel to model the data set after noise reduction by the filter.According to the calculation method of the loss function of the relation extraction model,the logarithmic probability value of each triplet is taken as the reward value to guide the Triplet Filter to retain high-quality samples.(4)The word vector is the feature of semantic information expression in text.This paper uses the word vector trained in the Baidu Encyclopedia corpus by word2 vec and the Bert word vector obtained through the Transfer Learning of relation extraction model.Experiments are performed on the Triplet Filter with two kinds of word vector,and the effects of different word vectors on the performance of the filter are compared.The Chinese relation extraction data set Du IE2.0 is annotated by the distant supervision method and contains a lot of noisy data.The experimental results on this data set show that the model proposed in this paper can retain the effective samples as much as possible while reducing the noise data,increase the sample utilization,and improve the performance of multiple relation extraction models.