Causal Relation Extraction Based On Knowledge Base And Pre-training

Nowadays,there is lots of data generated in the internet every day,the way about effectively classifying the data and their relationships is particularly important.Relation extraction as a main research of natural language processing,is often used for structured extraction of information.Causality is an common type of relation,it can be applied to a large number of fields such as smart medical,relational reasoning,and knowledge questioning,which can make our lives more convenient.Therefore,the method about extracting the causality in the text has become an pressing question to be worked out.Conventional relation extraction algorithms have problems such as defining complex extraction rules,large amount of annotations,and poor accuracy result.Deep learning has been proverbially applied due to the improvement of computer performance and capacity last few years.The emergence of Recurrent Neural Networks and Long Short Term Memory later,combined with word embedding generated by word pre-training,can extract characteristics from text very well.Therefore,it is more and more commonly used in relation extraction,which has reached a very great consequence.In 2018,Google put forward the BERT algorithm depended on the Transformer framework,which achieved the dynamic generation of the word embedding from the text,making the experimental consequence better.However,the BERT pre-training model is trained on an opening corpus,so it is short of certain technical information.Thus,training technical corpus requires a large number of computing resources and time,which is achieved hardly in normal laboratories.This paper proposes a causal relationship extraction method which combine BERT and knowledge base,applying the feature of BERT to actively generate word embedding from circumstances.Through combining the causal knowledge base,,this paper propose a C-BERT model,input the word embedding yielded by the C-BERT model into the Bi LSTM+CRF layer to complete the extraction of causality.This paper's major contributions are: 1)Set up a causal relationship knowledge base;2)Propose the C-BERT+Bi LSTM+CRF model,which combines the causal knowledge base with deep learning algorithms and applies it to the field of causal relationship extraction;3)Based on the Sem Eval dataset,this paper annotated 6128 new sentences,which contain a cause-effect relationship were selected and relabeled as CDS(Causality Dataset based on Semeval)dataset.Finally,the results of experiment illustrate that the CBERT+Bi LSTM+CRF algorithm has enhanced precision,recall and F1 results compared with the main model.