Entity Relation Extraction Method Based On Deep Transfer Learning

With the rapid development of the Internet,the amount of information available to human beings increases exponentially.How to mine effective information from massive data becomes an urgent problem.The research of information extraction is just produced in this situation.Its goal is to process unstructured information in natural text in a structured way,store it in a unified form,and integrate the acquired knowledge.Entity relation extraction is one of the most important parts.By understanding the text semantics and extracting the real relation between entities in the text,the triple of<entity-relation-entity>is formed as the most basic component of knowledge graph.Therefore,the accuracy and applicability of entity relation extraction technology directly affect the accuracy and volume of knowledge graph,which is very important for building large-scale knowledge graph.Deep learning model promotes the development of entity relation extraction technology,but it still faces the problems of scattered text data content,complex entity relationship categories,and difficult annotation.At present,the traditional task of entity relation extraction is to automatically generate a large number of annotation data through the distant supervision mechanism,and to design a deep neural network model for end-to-end entity relation classification.However,due to the influence of noise and mislabeling,the parameter updating in the model training process is affected by the error information,and the extraction accuracy is reduced.There are few-shot learning scenarios in biology,medical and other professional fields,so it is usually difficult to generate data automatically,and only a few or even single digit samples can be used.The conventional deep learning model is difficult to learn the multi-level mapping from text to classification in the case of a small number of samples,and the loss function cannot converge,resulting in the prediction results tend to be random and unable to be applied.In order to solve the above difficulties,this paper introduces deep transfer learning technology into the field of entity relation extraction.This paper relies on the strong feature extracting and fitting ability of deep neural network model,and uses the transfer learning method to apply the external knowledge obtained from the task with sufficient semantic information to the target task to assist learning,which improves the performance of entity relation extraction model.Firstly,in view of the noise scene under the distant supervision mechanism,this paper proposes an entity relation extraction model(BERT for Relation Extraction,BRE)based on the Bert pre-trained model,which pre-trains in the external corpus,and transfers the model parameters to the distant supervision entity relation dataset for fine-tuning.BRE constructs a position enhanced convolutional layer to enhance the processing of entity position information,and transfers the absolute position information of external corpus to the relative position information of entities,so that external knowledge can be integrated into the judgment of entity relation,and reduce the interference of noise to the real relation attribute.In addition,the time-decay selective attention mechanism is also designed to reduce the noise of multi-instance learning mode.In the process of model iteration,the low confidence instances are covered by time decaying,and the high confidence instances are retained to obtain the representation vector,which can alleviate the influence of wrong instances on gradient updating and improve the accuracy of the model.BRE was evaluated in NYT-10 and GIDS open datasets,and its superior performance was verified.Then,in the task of few-shot relation extraction,aiming at the problem that the model cannot converge due to the extreme scarcity of data,this paper proposes a few-shot relation extraction method based on metric-based meta learning.This method extends the idea of knowledge transfer,extends the transfer of model parameter knowledge to the transfer of category knowledge,that is,maintaining a measurement space,designing meta tasks to learn the matching relationship between instances and relation categories in space from sufficient data,and transfer the general category knowledge to the few-shot categories.In this paper,we design a few-shot entity relation extraction model,which is based on the deep neural network model and trained by the metric-based meta learning.It is called multi-channel attention network(MCAN).MOAN uses multi-channel parallel processing to retain more comprehensive information,and introduces double target auxiliary loss function to assist training.The performance and stability of MCAN framework are verified in NYT-10 and FewRel open datasets.