Research On Biomedical Event Extraction Based On Pre-trained Language Model

Studying the complex network of relationships between biological entities plays an important role in revealing the mysteries of organisms,and this knowledge information often exists in biomedical literature.How to automatically mine useful structured information from a large number of biomedical literatures has become an important research topic.Biomedical events focus on the specific behavior of biomedical molecules,and define more fine-grained complex relationships than coarse-grained binary relationships,which are of great significance for drug development and disease prevention.Biomedical event extraction has gradually become a research hotspot.The purpose of biomedical event extraction is to identify event trigger and related arguments.In recent years,biomedical event extraction has received more and more attention,methods based on rules,traditional machine learning and neural networks have been proposed successively.However,due to the phenomenon of nested medical events and the fact that the same trigger word may have multiple event types,the event extraction performance of the existing methods is not very high.To this end,this thesis divides event extraction into three stages: trigger recognition,arguments detection and post-processing,and proposes a new event extraction method.Considering that the pre-trained language model can effectively solve the problem of polysemy,and can generate a variety of semantic representations for the same trigger word according to different contexts,two different strategies are used for applying the pre-trained language model to trigger recognition.First,use a pre-trained language model to extract deep contextualized word representations,and then combine the deep contextualized word representations with traditional pre-trained word embeddings into a bi-directional long short-term memory network,and then label them with conditional random fields;the second strategy is to use a pre-trained language model for fine-tuning,the classification results can be directly processed with softmax,can also be labeled by CRF or BiLSTM-CRF.After the trigger recognition has been completed,and then arguments detection is performed.This thesis proposes twomulti-classification models for arguments detection.The first model is based on self-attention and entity attention.Self-attention focuses on different important parts of a sentence,the output of the self-attention is encoded with Bi-LSTM,and combined with entity attention,the semantic information of the sentence can be fully obtained;the second model utilizes the pre-trained model to encode the context information and candidate pair information,and the deep semantic information obtained is more abundant;these sufficient or rich semantic information facilitates the classification of complex relationships.After the trigger recognition and arguments detection are completed,this thesis uses rule-based post-processing method to generate events.Experiments on the MLEE dataset show that the method proposed in this thesis effectively improves the performance of event extraction,especially the extraction of complex nested events.