Research On Deep Learning-Based Key Techniques Of Fine-Grained Event Extraction

In the era of big data,text information,e.g.,social hotspots,entertainment news,policy announcements,and netizens’ blog sharing,has increased explosively,and has far exceeded human reading ability.But capturing and utilizing the important event information in these massive texts is of great value for many applications,e.g.,news hotspot tracking,public figure supervision,policy invest-financing,and figure portrait in social network.Therefore,automatic text event analyzing technology has attracted extensive attention.There is also a specialized event extraction task in academic circles.As an information extraction task in the field of natural language processing,event extraction is required to identify the types and participants(also known as "arguments" or "argument entities")of events described in a given text automatically.Although type information and arguments are core elements of restoring events,such task setting neglects the relationships between arguments,hinders in-depth exploration of event details,and thus limits the practicability.In light of this,we introduce another information extraction technique--"entity relation extraction" to distinguish argument relation,thereby further supplementing event-related details in the extraction results.We define the event extraction with argument relation as Fine-Grained Event Extraction,and split it into three key techniques:event extraction,event argument extraction,and entity relation extraction,which are also three important research tasks.With the rapid development and popularization of deep learning technology in recent years,the above three tasks have made great breakthroughs by various neural networks.As a data-driven technology,the state-of-the-art(SOTA)neural networks on these tasks are all based on supervised learning,which are also the main study objects of our work.Our innovations and contributions are summarized as follows:(1)We propose an event detection neural network integrating global statistical information.Event detection task aims at identifying event types described in given texts and locating their triggers.Existing event detection neural networks mainly adopt a sequence labeling manner,and judge triggers according to the context of each word.But in reality,the supervised training data is very limited,and the models is difficult to learn various context combinations sufficiently,so that they are prone to "get lost" in the new texts in testing.Through statistics,we reveal that the co-occurrence frequencies of most words with all events have clear directivity and well stability in realistic event detection datasets,which can greatly narrow down the choices of event detection models in decision-making.Based on this discovery,we propose a Semantic and Statistic-Joint Discriminative Network(SS-JDN),which converts word-event occurrence frequencies collected from the global training set into a set of statistical event features in both direct and indirect ways,and then combines them with the contextual features to form the final decision basis.The experiments on two public datasets show that SS-JDN effectively outperforms 10 SOTA event detection contrast models by 1.9%on micro F1 score.(2)We propose a dual-expert argument extraction neural network enhanced by meta learning.Argument extraction models are required to find out the arguments of events described in the given texts and classify their roles.We observe that the instance numbers of different roles present an obvious long-tail distribution in realistic argument extraction data,which shows "imbalanced instance numbers between ’head’ and ’tail’ roles"and "scarce instances of ’tail’ roles" two characteristics.On the other hand,the widely-used deep learning methods for long-tail data cannot avoid the sacrifice of performance on head roles when improving the tail ones.To this end,we propose a dual-expert argument extraction neural network,which separates the predictions of head and tail roles and assigns them to a head expert module and a tail expert module for executing,thereby avoiding their mutual effect.In inference,each encoded instance will be allocated to one of the two experts by a routing mechanism.Considering the above two characteristics of long-tail data,we propose a Balanced Routing Mechanism to reduce routing errors caused by the imbalance of role instances,and design a novel Target-Specialized Meta Learning(TSML)algorithm to facilitate the learning of the tail expert with scarce training instances.The experiments on two public datasets demonstrate that our dual-expert model enhanced by TSML significantly outperforms the SOTA event argument extraction models and advanced generic methods for long-tail training data by 3.1%and 3.4%on macro F1 score.(3)We propose a distantly supervised relation extraction neural networks obtained by a Multi-instance Dynamic Temperature Distillation framework.The relations between different argument entities need the relation extraction technique to distinguish.To break the limitation of manually annotated training data,we focus on distantly supervised relation extraction technique,which can annotate sufficient training data for relation extraction models automatically while brings lots of wronglyannotated noise instances.It has been demonstrated that "teacher-student"training framework of knowledge distillation method can alleviate the interference of noises by softening the student model’s target distributions with the teacher model.However,the conventional knowledge distillation method suffers from two problems of "propagation of inaccurate dark knowledge" and "constraint of a unified distillation temperature".We propose a simple and model-agnostic knowledge distillation framework,which introduces Multi-Instance Learning technique to alleviate the propagation of inaccurate dark knowledge and adopts a Dynamic Temperature Regulation technique to break the constraint of unified distillation temperature,thereby softening more student’s targets to a moderate range.We distill three types of distantly supervised relation extraction neural networks on two public datasets,whose micro F1 score effectively exceed their teachers and the SOTA contrast models by 2.0%.