Relation Extraction Based On Dualchannel Attention And Pre-Trained Language Model

Today's society is an information explosion society,The information industry is entering a period of rapid development.The information society makes communication between people more convenient.People's life with e-commerce,mobile Internet and other computer technology development have become richer and richer.Information on the Internet has important value.To obtain this important information from the Internet,so we need information extraction technology.Relationship extraction is the automatic extraction of inter-entity relationships from large-scale unstructured text,which is an important part of information extraction,which has been widely concerned.Relationship extraction based on deep learning is the mainstream method at present.By constructing neural network model and using a large number of text data for learning,the performance of relationship extraction is greatly improved.However,there are still two problems.First,most of the models are based on RNN or CNN.The disadvantages of RNN and CNN lead to incomplete feature vectors.Secondly,the static word vector is weak in the semantic expression of the text,unable to express the phenomenon of polysemy.In order to realize the extraction of important features in text,this paper constructs a relationship extraction model based on dual channel fusion of Bi GRU and CNN.Because the relationship extraction mainly focuses on the features of entity words in sentences,CNN model can extract the features of entity words and their surrounding words well.However,when the sentences are long and the distance between entities is long,the relationship extraction model can extract the features of entity words and their surrounding words,There is no good connection between the features of two entities.Therefore,this model combines Bi GRU and CNN in a dual channel way.Bi GRU is used to extract the long-distance dependence features of entity pairs in the text,and CNN is used to extract the local dependence features between words.Then,the model selectively focuses on the important parts of the input sequence through the attention mechanism to remove the noise interference of non entity words in the sentence.Finally,softmax classifier is used to extract the local dependence features,Complete the task of relationship classification.The experiment is based on semeval-2010 task8 data set,and six baseline models are selected as the control.From the experimental results,we can know that the model in this chapter does have better ability of relationship extraction.Aiming at the disadvantages of static word vector,a relation extraction model based on BERT language model is proposed.Traditional relation extraction methods use static word vectors such as word2 vec,which leads to the invariance of word vector representation in different contexts,resulting in the loss of semantic information.In this method,the word features are extracted by BERT,and then the context semantic features are extracted by Bi GRU network.Finally,the feature vector is used for feature classification.In the experiment,11 representative models were selected as the contrast on semeval-2010 task 8 data set.According to the experimental results,the F1 value of the model in this chapter is greatly improved compared with other models.A relationship extraction system based on deep learning is constructed,which provides entity recognition,relationship extraction and other functions.Django is used to develop the web application background,and bootstrap is used to make the foreground page.The performance test shows that the system has realized the function of relation extraction and has good concurrent access ability.