| In recent years,with the rapid development of web technology,we have gradually entered the era of big data.A large amount of text data has appeared in some specific fields.These text data contain rich textual information,and how to extract valuable structural information from these unstructured text data is an important sub-task in information extraction.The main purpose of this task is to extract entities and relations from the text and generate the corresponding triples,i.e.(subject,relation,object).However extracting entities as well as relations between entities is difficult and practically complex to achieve under numerous factors.This thesis proposes a joint extraction model that combines entity extraction with relation recognition,which is based on neural networks,BERT pretrained language model,and multi-headed attention system.Using entity extraction and relationship extraction models,this model effectively combines entity extraction and relation extraction in order to extract entity-relationships from unstructured text effectively and achieve information extraction that is effective.The main research contents of thesis are as follows.(1)A brand new entity recognition model and a relationship extraction model based on the BERT model are proposed.Based on the BERT model,the word embedding vector of the text is firstly obtained.After that the normalization process is done for the sequence.Each token in the sequence is identified by multiple classifications to get the classification that the entity belongs to the subject or object.The purpose of the relationship extraction model is to extract all possible classes of relationships embedded in the text without requiring information about related entities.The model averages the embedding vector representations of individual utterances to obtain vectors that can represent the overall information of the text,and extracts the possible relationship categories contained in that utterance through a neural network-based classification model.The F1 score was mainly used to evaluate the prediction results of the model in these two tasks.Compared with the existing models,the F1 score of this model for entity extraction and relationship extraction results reached 94.9% and 95.5%,respectively,which were both better than the comparison algorithms.(2)A fusion model AJE based on the multi-headed attention mechanism is proposed,which can jointly train the above two models so that the two models share network parameters and extract the triples simultaneously in one stage to achieve the purpose of joint extraction.In order to fuse the models,the subjects and objects in the entities are first extracted,which are regarded as query and key vectors in the attention mechanism,respectively,and the relationship categories are regarded as value vectors.After that,these vectors are processed in the multi-head mechanism so that the model can learn information about different subspace representations in the text.Finally,all queries,keys and values are traversed and attention scores are computed to obtain the correctness of the corresponding triples.In thesis,experiments were conducted on the NYT dataset and compared with the existing joint extraction model,and the final F1 scores reached 92.7%,all better than the control model.It also has a better performance for problems of various complex cases due to traversing all entities and relations. |
