Research On Relation Extraction Based On Deep Neural Networks

In the information age,a lot of text information is produced every day.Information extrac-tion plays an increasingly important role in the field of natural language processing.As an important research topic of information extraction,relation extraction is a key step in the process of knowledge map construction and completion.In recent years,the construction of relational extraction model has developed from the ini-tial feature engineering to neural network and deep learning.Through the combination of algorithms in image processing,voice processing and other fields with relational extraction tasks,fruitful research results have been achieved.Att+BiLSTM model is one of the more breakthrough work.Through the application of attention mechanism in the bilstm network,combined with the ability of bilstm to obtain long-distance dependent information in long sentences and the ability of attention mechanism to obtain important words and advanced semantics of sentences,the state of art performance is achieved without additional artificial features.However,the whole attention mechanism obtains the attention degree of each word corresponding to the whole sentence,and does not focus on the essential entity part of the relation extraction task.At the same time,some keywords in the non entity part of the sen-tence that determine the high-level semantics of the sentence are not better to identify and extract the high-level semantics.Therefore,in view of the task of relation extraction and the understanding of relation extrac-tion model,this thesis proposes that the relation extraction model should have the ability to recognize the entity pairs and the whole sentence high-level semantics,and according to the proposed ability,puts forward a new relation extraction model and corresponding methods to improve the ability of relation extraction model to complete the task.(1)In the traditional relation extraction model,in order to increase the ability of identifying entities,the general approach is to indicate the location of entities through the embedded layer position indicator(PI).However,such an approach will lead to data reduction after embedding through neural network.In order to improve the ability of identifying entity pairs,a relation extraction model based on entity attention is proposed.In this model,the method of adding entity pair attention is used to enhance the recognition of entity location in the model,so as to increase the attention of the model to the advanced semantics of entity pair.First,use bidirectional LSTM to model the language context of entities.Secondly,the attention mechanism of entity is added to construct the independent attention degree of each entity in the model,and the attention of two entities is combined by the method of adding,so as to increase the understanding of each entity's high-level semantics and location information in the model.Then,softmax is used to normalize the calculation results of related features of different entities into the probability of all semantic relations.Finally,the gradient descent algorithm is used to optimize the model parameters.The experimental results show that this model has better performance in relation extraction task than the traditional bilstm model combined with attention.(2)In order to solve the problem that the model can complete the task of relation extraction when the entity semantics is unknown,and increase the ability of the model to recognize the advanced semantics of sentences outside the entity,a new sentence attention mechanism is proposed based on the entity attention.In this model,the entity part of the sentence is replaced by the blank part.By this way,the entity pair of the sentence is lost,which forces the model to learn the non entity part of the sentence to increase the understanding of the high-level semantics of the whole sentence.And in order to combine entity attention with sentence attention better,improve the combination of attention level.Finally,through comparative experiments,it is proved that the combination of sentence attention and entity attention can better improve the ability of the model to complete the task of relation extraction.