Design And Implementation Of Continual Relation Extraction System Based On Memory Replay

As an important part of information extraction,relation extraction is widely used in knowledge graph construction,question answering system and other fields.It mainly extracts structured information from unstructured plain text information,which is convenient for human and machine processing,and generally consists of tuples containing entity words and relational words.Traditional relational extraction only pays attention to the learning effect of models in static data sets with predefined relationships,but with the continuous increase of information,how to learn models in dynamically increasing data sets becomes a new challenge for relational extraction.This leads to the task of continual relation extraction,that is,the relation set that needs to be extracted by the model will expand with the accumulation of data,and the storage space and computing resources will limit the use of the whole past training data.As a result,the existing continual relational extraction system is faced with two problems in engineering application: the discovery of new relation and the learning of model.With the development of deep learning technology,the deep semantic features of pre-training language models generally improve the effect of natural language processing tasks.However,in the new relationship discovery,because there is no predefined relationship type,the pre-training model adopts the conventional pre-training fine-tuning paradigm to learn,which will introduce additional network structure and make it difficult to effectively cover the open data scene.In continual learning,the existing regularization method,dynamic network method and memory system method are excellent in image classification,but they are poor in the field of natural language processing.Among them,the research of continual relation extraction based on memory system still has some problems,such as catastrophic forgetting and memory set over-fitting.Aiming at the problems of the above-mentioned continual relation extraction system,this thesis proposes a relation type inference method based on prompt learning and context awareness,and a continual relation extraction model based on memory replay,and establishes a highly usable relation extraction system.The main work of this thesis is as follows:(1)A relation type inference method based on prompt learning and context awareness is proposed.In this method,the open relationship recognition task is modeled as a prompt learning problem,and a hand-designed prompt template and a knowledge-enhanced verbalizer are introduced.Furthermore,the semantic information of context is fused to cluster the relative words unsupervised.With limited human participation and no additional parameters in the pre-training model,better results are achieved.(2)A continual relation extraction algorithm based on memory replay is proposed.In this algorithm,sample representation is used to sample the examples,and fix model classifier to adjusts the representation of the samples,then the pseudo samples with higher robustness are obtained as the memory set.While learning the new relationship,the distillation loss function is added for the memory set to restore the classification ability of the model for past tasks,which improves the performance of the model in processing the task sequence of continual data sets.(3)A website system of continual relation extraction based on memory replay is built.The system can manage users’ training and predicting tasks,datasets and different models in the scenario of continual relation extraction.The system can extract information from unlabeled data sets and generate new relations set.Users can use the annotation sets generated by the system or added by themselves to execute the training and predicting of continual relation extraction models.