Eventic Graphs Construction And Application Methods For Textual Event Prediction

Eventic knowledge refers to the evolutionary principles and patterns between events,which following specific temporal and spatial orders,including the sequential,causal,conditional,entailment,and hierarchical relations.This kind of knowledge is very valuable.Mining it from data is very meaningful for people to understand the principles of human behavior and social development,and for the development of artificial intelligence applications.However,the existing large-scale knowledge bases,especially the knowledge graphs,generally focus on entities and their relations,and fail to reveal the evolutionary principles and patterns of events.In order to make up for this shortcoming of the traditional knowledge graphs,this article shifts the research focus of knowledge graphs from nominal entities to verbal events,thus proposing the concept of eventic graph.As a new form of knowledge representation,Eventic Graph(EG)is a knowledge base that describes the evolutionary principles and patterns of events.Structurally,Eventic Graph is a directed graph,in which nodes represent events,and directed edges represent the logical relationships between events,such as the sequential,causal,conditional,entailment,and hierarchical relations.This paper starts with the basic concepts of the eventic graph,explores the eventic graph construction method by extracting event relations from texts,proposes an end-toend neural network-based causal generation method,and explores the application method of eventic graph in textual event relation inference tasks,such as event prediction and causal relation classification.Specifically,the main content of this research includes the following four aspects:1.Eventic Graphs Construction by Textual Event Relations Extraction.This thesis proposes the eventic graph and describes the basic concepts of eventic graph.This thesis also proposes a method framework for constructing eventic graphs.The experimental results show that the proposed eventic graph construction method has great potential and can effectively extract the sequential relations from texts.This thesis constructs a Chinese travel domain eventic graph,which contains nearly 30,000 event nodes and more than 230,000 directed edges.For financial domain,we construct a causal eventic graph,including 2.19 million event nodes and 1.61 million causal edges.2.Eventic Graph Completion by Causal Knowledge Generation.This research explores the causal knowledge generation technology based on end-to-end neural networks.For any open-domain sentence-level event input,the system can generate multiple possible causes and results.We develop a large-scale English causal dataset(CausalBank)for training the model.In addition,we have also extended the keyword constrained decoding for text generation to support disjunctive positive constraints decoding.Manual and automatic evaluation metrics show that our method can generate high-quality,multi-semantic causes and effects even for brand new inputs.The acquisition of causal knowledge by generation is an important supplement to extraction.3.Story Ending Prediction with Transferable Pre-trained Language Models.This research explores how to combine event relations knowledge with pre-trained language models and inject event knowledge into the pre-trained language model when a large number of event relations have been obtained,thereby helping to improve the performance of downstream tasks such as event prediction and causal inference.We propose a three-stage training framework called Trans BERT.It can not only learn general language knowledge from large-scale unlabeled data,but also effectively utilize the supervised information provided by various semantic-related tasks.Experimental results on multiple datasets including story ending prediction and causal pairs classification show that Trans BERT generalize well to other tasks,languages,and pre-trained models.4.Script Event Prediction with Scaled Graph Neural Network.The eventic graph itself is a graph-like structure.If there is already a constructed eventic graph,this research explores how to perform event reasoning and prediction directly on the eventic graph structure.Specifically,we propose a scalable graph neural network model,which combines large-scale eventic graph with end-to-end graph neural network training method by performing network representation learning and reasoning on only one subgraph structure at a time.Based on the eventic graph and graph neural network,it has shown superior performance on the script event prediction task,indicating great potential of the eventic graph in event prediction.In general,this thesis has carried out research on the extractive construction of eventic graphs,the generative completion of causal knowledge,and the application method of eventic graph in textual event relation inference tasks.The research in this article is only a preliminary exploration in the research direction of the eventic graph,and it is hoped that this research can inspire more future work on the research and development of the eventic graph.