Research On Entity Relation Extraction Method Using Pre-training Language Model And Knowledge Representation

Entity Relation Extraction is an important sub-task of Information Extraction.Its research results provide necessary technical support for many fields of Natural Language Processing,and have important significance and application value.With the development of deep learning,the entity relation extraction process no longer needs to manually construct features,but automatically extract features from the data set through the neural network model.However,due to the complexity and high cost of manual labeling,it is not possible to obtain a large number of entity relation labeling data sets.Therefore,it is difficult for us to learn effective entity relation features from sparse data sets.On the other hand,most of the Entity Relation Extraction methods rely on adding external resource features such as Word Net and POS.The accuracy and validity of these features largely determine the upper limit of the model algorithm.In view of the above problems,this paper proposes a corresponding solution to improve the feature representation ability of the entity relation extraction model in sparse data and reduce the dependence on the characteristics of external resources.The main research contents and innovations are as follows:(1)The Entity Relation Extraction frame based on pre-training language model BERT is constructed as the baseline model.This model takes sentences as input sequences without any input features and obtains vector representation with deep semantic information through Bert pre-training language model.(2)On this basis,the Bidirectional Gating Recurrent Unit network model which integrates pre-training language model BERT and attention mechanism(BERT-Att-BiGRU)is proposed.The sentence representation with deep semantic information obtained from the Bert model is input into the Bi-GRU network frame to further obtain high level semantic information.An attention mechanism is added after Bi-GRU to obtain more useful features for relationship prediction.On the other hand,Bidirectional Gating Recurrent Unit network model which integrates Bert and Max Pooling(BERT-Max Pooling-Bi-GRU)is proposed.Different from BERT-Att-Bi-GRU,the features of BiGRU output are selected through Max Pooling operation to obtain the global semantic features of the sentence.The experimental results show that the two models proposed in this paper achieve better performance than baseline model,which fully proves the feasible and effective of integrating the Bert pre-training language model into the Bi-GRU network frame.(3)The Entity Relation Extraction method which integrates Knowledge Representation is proposed.This method can make full use of the knowledge triple information in the sparse data,it can obtain the relation representation with implicit semantic features in the sentence through the knowledge representation learning method.Finally,establish the joint learning of relation representation and sentence representation from text information.The experimental results show that our method has more advantages than the previous method which only uses text information.The comparative experiments show that the two Entity Relation Extraction methods proposed in this paper,which integrates pre-training language model and integrates knowledge representation,effectively improves the feature representation ability of the model.In particular,the method of fusion pre-training language model proposed in this paper,performance better than previous neural network model methods without additional features,which fully proves the superiority of the method of using pre-training language model.