Research On Event Extraction Based On Tri-training And Pre-trained Model

Event extraction is the current research hotspot and difficulty in the field of natural language processing.The purpose is to extract important event information reflecting objective facts from large-scale,unstructured natural language texts.It has important application value in multiple directions such as intelligent question answering,automatic text summarization,and knowledge graph construction.Event extraction modeling is the core issue of event extraction research.Traditional statistical learning and end-to-end deep learning adopt supervised learning strategy for event extraction modeling,which is susceptible to the labeled data that is the small scale and sparse,and there exists the error propagation problem based on the pipeline scheme modeling.Therefore,in response to the above-mentioned event extraction modeling problems,this dissertation proposes to transform event extraction modeling into sequence labeling modeling,and conducts research on event extraction from the perspectives of tri-training and pre-training.The main research work is as follows:(1)Event extraction based on feature template selection and tri-training.The method employs the joint-learning modeling scheme to build a joint labeling model of triggers and event arguments based on conditional random field(CRF).Firstly,multiple feature templates are constructed based on the features of the training set,and the optimal feature template is selected according to the results of the ten-fold cross-validation.Then,the tritraining algorithm is applied to sequence labeling modeling,its initial setting is improved,and a mechanism to determine the label sequence of unlabeled samples is designed,so as to ensure unlabeled samples with high confidence are added to the labeled training set to expand the experimental data.Finally,Tri-training-CRF model is built based on the optimal feature template and the tri-training algorithm to complete the joint labeling task of triggers and event arguments.The experimental results show that Tri-training-CRF can effectively alleviate the problems of data sparseness and error propagation.Compared with traditional supervised statistical learning methods,Tri-training-CRF has better event extraction performance.(2)Event extraction based on pre-training BERT deep neural network.Aiming at the problem that statistical learning methods modeling is restricted by feature engineering and unable to extract deep contextual features of text,the method introduces BERT and Bi LSTM to build a joint labeling model of triggers and event arguments,that is,BERTBi LSTM-CRF.Specifically,the joint labeling model is initialized with the model parameters of the pre-trained BERT,and each character is transformed into a character vector containing rich contextual semantic information.Then,the BERT character vectors are fed into a Bi LSTM to further capture the contextual features,so as to obtain the potential semantic relations between triggers and event arguments.Finally,the output of Bi LSTM is processed through a CRF to generate the best label sequences of triggers and event arguments.The experimental results show that BERT-Bi LSTM-CRF can not only effectively avoid the troubles of feature engineering,but also solve the parameter learning problem of end-to-end deep learning methods due to the labeled data that is less and sparse.