Causal Relation Extraction Based On Semantic Dependency Parsing And Pre-trained Language Models

Causal relation extraction is an important research direction of natural language processing,the existing research transforms causal relation extraction into relation classification or sequence labeling tasks.Causal relation in the text has rich expression form,for the complex causal relationship in the sentence and the document-level causal relation,the existing methods are difficult to extract effectively.In addition,most of these studies ignore the exploration of the semantic information between causal entities.In response to the above problems,this paper introduces semantic dependency parsing and pre-trained language models,combined with deep learning,two causal relation extraction models were proposed,which can effectively extract various causal relation from the text.The specific contents are as follows:(1)Causal relation extraction based on semantic dependency graph and BERT in sentences:Due to the existing sequence labeling model lacks the selection of causal feature,this paper proposes a causal relation extraction model:BERT+Bi-LSTM+SDGAT+CRF.Use the pre-trained language model BERT to obtain sufficient causal semantic features from the text,use semantic dependency graph and attention mechanism to enhance the causal information in the text,and weaken other useless information,get the causal label of each word in the sentence.In addition,this paper summarizes the characteristics and shortcomings of the existing public sentence-level causal datasets,recreates and annotates a sentence-level dataset SCE with comprehensive causal types.(2)Causal relation extraction based on semantic dependency parsing and pre-trained word embedding in document: In order to further study the causal types across sentences and paragraphs,this paper extracts the causal relation from the document based on relation classification task.Use semantic dependency parsing to supplement causal entity pairs for the documents,word embedding of pre-trained language models are used as input,and causal relation in entities is extracted from documents in combination with the basic model of deep learning.In addition,the English document-level causal dataset Doc EEC and the Chinese document-level causal dataset Doc CEC were built.In this paper,a large number of experiments have been done on the sentence-level causal dataset SCE,the Chinese and English document causal dataset Doc CEC and Doc EEC,and a variety of common evaluation indicators are used for verification.The experimental results show that the method proposed in this paper can improve the results of causal extraction,which proves that the pre-trained language models and semantic dependency parsing have advantages in strengthening the causal semantic information.The F1 score and accuracy of the sentence causal relation extraction model proposed in this paper are relatively high,which proves the effectiveness of this method to deal with the complex causality in the sentence.In addition,the document-level causal relation extraction model proposed in this paper has achieved the expected goal in the experiment,which proves that the method can extract the causal relation across sentences and paragraphs,but the extraction of causal relation in the document is still a complex problem,this work provides a direction for subsequent related research.