Research On The Key Methods For Joint Extraction Of Entities And Relations Based On Deep Neural Network

With the development of Internet technology,the world is moving toward the era of big data.How to extract useful information from massive text data accurately and efficiently becomes more and more important.Relation extraction,as the main task of information extraction,aims to extract the semantic relations between entities from unstructured text.It has significant practical application in the fields such as constructing knowledge graphs,information retrieval and automatic question answering.The traditional approach generally regards relation extraction as an independent task and adopts the pipeline method,that is,classify the relations based on the results of named entity recognition task.Its main disadvantage is that it ignores the relevance between the two tasks of named entity recognition and relation extraction,and causes the propagation of errors,thus affecting the final effect.Therefore,the joint extraction technology of entities and relations has begun to attract widespread attention from academia and the business community and has gradually become a hot research topic.Although the joint method can extract the entities and the relations from the text directly,it also introduces a more complex model structure.At present,the research has just started and there are still many problems to be solved.This paper mainly focuses on the joint extraction of entities and relations based on sequence-to-sequence method,and by analyzing the existing problems and proposing improvements.The main research work in this paper is summarized as follows:First,through the analysis of the copy mechanism of the sequence-to-sequence method,it is found that the defect of the scoring function will lead to instability in the entity copying.Aiming at this problem,this paper proposes a joint extraction model of entities and relations based on improved copy mechanism.By using the newly designed scoring function,the decoder of the model can solve the problem that the softmax function is not applicable due to the same probability distribution when copying the two entities,and avoid the high dependence of the model on the mask,thereby improving the model's extraction performance.The experimental results show that the model has significantly improved performance on the joint extraction task of entities and relations compared with the baseline.Second,by analyzing the output results of the sequence-to-sequence method,it is found that for an entity composed of multiple words,only the last word of the entity can be extracted as the output result.Aiming at the above entity integrity problem,this paper proposes a joint extraction model of entities and relations based on multi-task learning.The model builds an additional conditional random field(CRF)layer at the output of the encoder for sequence labeling tasks.The additional layer can mark the entities in the input sentence to assist the model in extracting the complete entity.The model trains the encoder and decoder simultaneously through the objective function of multi-task learning.Experiments results show that the model effectively solves the integrity problem of the entity and achieves performance improvement.For the traditional static word embedding,Word2vec model cannot express the polysemy of the word widely existed in natural language,this paper proposes a BERT-based joint extraction model of entities and relations.By introducing BERT dynamic word embedding representation based on context as input to the model,it provides more accurate semantic feature information for this model.The experimental results show that the BERT word embedding representation can significantly improve the performance of the joint extraction model.