Document-level Causal Relation Extraction Based On Pretrained Language Models And Graph Convolutional Neural Networks

With the rapid development of Internet technology,people in the online world generate massive amounts of data every day,and people hope to discover useful information from the massive amounts of data.In the massive information,text data occupies an important position,and relation extraction from text data is an important method to study text data.Causality is an important research direction in relation extraction.It plays a vital role in many fields such as medicine,economics,question answering systems,and public opinion monitoring.The essence of most problems in scientific research is to explore the relationship between things.The 2021 Nobel Prize in Economics is awarded to scholars who conduct causal inference in the field of economics.In recent years,the strong rise of deep learning models has profoundly affected the research methods of natural language processing.As an important branch of natural language processing,relation extraction has also undergone changes in its research process.The methods and models based on deep learning require large-scale pre-training corpus to support the training and optimization of the model,so the massive text data is very suitable for the characteristics of deep learning models.Causal relationship extraction plays a very important role in mining textual information,and causality can help computers reach or surpass the cognitive level of human understanding of language and words.At present,the mainstream method in the field of causality extraction is still sequence labeling.Its main advantage is to transform the abstract natural language understanding problem into a mathematically solvable probability prediction problem.However,the current causal relationship extraction data set under sequence labeling is very limited.The main research object of the existing causal relationship extraction is the causal relationship in a single sentence,and the extraction target is the words or phrases that express the causal relationship in the sentence,but only isolated words or phrases cannot fully express a causal relationship,which is easy to cause causality.Lack of information;current research lacks due attention to extracting causal relationships across sentences.In causal relationship extraction,word vectors are generated by a large-scale pre-training model(such as the BERT model).The BERT model only pays attention to the local context information of the sentence,ignoring the global information in the document.The global information in the document may play a more important role in relation extraction.In response to the above problems,this thesis introduces the concept of clauses and extracts causal clauses to obtain a causal expression as complete as possible.Through the combination of graph neural network and large-scale pre-training model,the model can capture the global information and local context information of the text at the same time,and extract the causal relationship clause.The main contributions of this thesis are as follows:(1)In the causal relationship extraction,this thesis changes the extraction granularity,expands the extraction object from words to clauses,and extracts the cause clause and the result clause,which makes the expression of causality more complete.(2)On the algorithm model,we extend the graph convolutional neural network and introduce the clause graph convolutional neural network(CGCN).Combining a large-scale pre-training model with a clause graph convolutional neural network,it is applied to the task of causality extraction and achieves good experimental results.(3)In terms of experimental data,based on the existing Weibo dataset,expand and correct the data set.