An Automatic Event Detection Model Based On Deep Context And Attention Mechanism

Event detection is an important subtask of event extraction,which aims to extract event triggers from individual sentences and further identify the type of the corresponding events.Now main difficulties of this task lie in polysemy and multi-event detection.Polysemy refers to the fact that the same word may trigger different types of events in different contexts.While multi-event means there may be more than one event in a sentence.To solve these two difficulties simultaneously,we propose a novel recurrent and convolutional network based on language model(ELMo)and attention mechanism.This model can not only learn the deep semantic information but also extract the chunk information and abstract features.Besides,the attention mechanism in model can give higher weights to relevant useful information and weaken the influence of noise information.What's more,the ELMo in model which is deep contextualized word representation can provide more grammatical and semantic information for the model and effectively solve the ambiguity of words.Then the performance can be improved through these strategies.Experiments in ACE2005 English corpus show that the final model achieves the state-of-the-art performance with Fi\value is 74.4%,which is better than the best performance of related work.The main research work in this paper includes following parts:(1)Firstly,we study a hierarchical neural network based on word2vec.This model contains Bi-directional long short-term memory neltwork(Bi-LSTM)and dynamic convolutional neural network(DCNN),so that we call this model as BiLSTM-DCNN.In this model,we explore the effects of different word vector dimensions,input characteristics and model structure in BiLSTM-DCNN,and take the best results as the basis of the following researches in this paper.(2)Secondly,we introduce the attention mechanism into BiLSTM-DCNN to explore the effects of different attention mechanisms on the model.Research shows that the new model(ATT-RCNN)performance can be improved by choosing right attention mechanism algorithm.(3)Thirdly,we propose a hierarchical neural network based on attention mechanism and ELMo(LM-ARCNN)to explore the effects of ELMo on the model.Experimental results show that using ELMo instead of traditional word embedding,such as word2vec can further improve the performance of the model.(4)Finally,we compare our three models to the related works.Experimental results show that our model can make significantly improvement in both polysemy task and multi-event detection task.