Sequence Tagging Causal Relation Extraction Based On Multi-task Learning

The causal relation extraction task is one of the important research tasks in the field of natural language processing,and it plays a crucial role in knowledge graphs,question answering systems,and event prediction.Most of the existing research methods are based on single-task learning,ignoring the connection between the causal relation extraction task and other tasks.In addition,the corpus contains complex causal relation such as one-to-many and many-to-many,and the number of non-causal entity pairs(negative instances)usually far exceeds the number of causal entity pairs(positive instances).Faced with the complex causal relation in the text and the scarcity of causal data,these research methods usually cannot achieve good extraction results.In response to the above problems,this paper introduces the parameter sharing mechanism of the multi-task learning framework.Through the parameter sharing between tasks,the causal relation extraction task can fully capture the hidden semantic information in the auxiliary task,and solve the problem of poor causal relation extraction effect in the unbalanced corpus.Through the BIO sequence tagging method,the causal relation extraction task is converted into a sequence tagging task to solve the problem of insufficient causal relation extraction in the corpus.The main research work of this paper is as follows:1.The interactive Transformer framework of slot filling and intent detection in the spoken language understanding system is introduced into the field of natural language processing,and a multi-task learning based on causal strength classification and causal relation extraction model(CIT-CSCR)is proposed.The label attention mechanism is introduced to obtain the explicit feature representation of the two tasks,and the bidirectional connection between the two tasks is established through the multi-task learning framework to explore the impact of multi-task learning on the causal relation extraction task.The experimental results show that the method proposed in this paper obtains rich semantic information in the causal relation extraction task through the interaction between the attention mechanism in the causal strength classification task and the causal relation extraction task.Compared with single-task learning methods,it achieves a significant improvement in performance.2.In order to further improve the performance of causal relation extraction,combined with the BERT pre-trained language model and multi-task learning framework,a sequence tagging causal relation extraction model(MTLSS+BERT+BiLSTM+CRF)based on BERT and sparse sharing mechanism is proposed.First,the BIO sequence tagging method is used to label the sample data,and the causal relation extraction task is transformed into a sequence tagging task.Then,part-of-speech tagging(POS)and chunk parsing(Chunk)are selected as auxiliary tasks for the causal relation extraction task,and a sparse sharing architecture is selected as the multi-task learning framework.Finally,the BERT pre-training model is introduced to obtain sufficient causal semantic information from the context,and the redundant parameters of the model are reduced by parameter pruning technology.Extensive experiments were carried out.The experimental results show that compared with the traditional single-task extraction method,the sparse sharing mechanism reduces the parameters of the model,promotes the interaction between tasks,and improves the performance of the causal relation extraction task,and its F1 value reaches the highest.The multi-task learning model proposed in this paper achieves the desired effect in the causal relation extraction task,improves the accuracy of complex causal relation extraction,and alleviates the problem of causal relation extraction from unbalanced corpora to a certain extent.The work of this paper provides a new idea for the subsequent research on causal relation extraction tasks.