Research On Implicit Discourse Relation Recognition Based On Graph Neural Network

Discourse relation recognition aims to identify the semantic connection relation between two discourse units(arguments)in the same text,such as comparison relation,temporal relation,etc.,which benefits to the upper-layer tasks of natural language processing fields,such as machine translation,question answering system,reading comprehension,etc.Penn Discourse Tree Bank provides authoritative linguistic resource support for the study of discourse relation,annotating five types of discourse instances,and defining a three layer discourse relation system.It divides the discourse relation into explicit discourse relation and implicit discourse relation according to whether there is a connective linking the two arguments.Since connectives contain obvious semantic information,only extracting the features of connectives can make the accuracy of explicit discourse relation recognition reach more than 90%.However,the implicit discourse relation recognition lacks obvious semantic features,requiring natural language processing algorithms to fully understand text semantics and infer semantic connection between arguments.Due to the shortcomings of existing methods in the deep representations of discourse arguments and the mining of interactive features between arguments,the current recognition accuracy is far from usable.In order to learn the argument representations suitable for discourse relation recognition and better integrate the argument itself and interactive semantic features,we propose implicit discourse relation recognition method based on the graph neural network.The specific work content can be divided into the following two points:(1)Implicit Discourse Relation Recognition based on Graph Convolutional NetworkThe representations of discourse arguments and the mining of interactive association clues between two arguments are the key issues in implicit discourse relation recognition.However,existing studies have shortcomings in the distributed representations of arguments such as insufficient training data,semantic ambiguity,and lack of contextual information.There are shortcomings in mining argument interactive patterns such as large computational granularity and difficulty in distinguishing keyword pairs.In order to solve the above problems,we build a static graph based on the dependency syntactic structure of argument pairs,embodies the interactive patterns of arguments in the graph structure,integrates the argument representations into the embedded representations of nodes,and uses graph convolutional network for graph representation learning,and then get the argument pair representation.Specifically,the syntactic dependency tree of discourse argument pairs is firstly constructed,the dependency relations between words is extracted from it,and a static graph containing syntactic information is constructed based on the dependency relation.This can shorten the distance of some keyword pairs,which is helpful for subsequent networks mining of semantic information in the graph.Then,the pre-trained BERT model is used to obtain the context-dependent argument representations,which is used as the node features of the static graph,so that the pre-training result of the language model on the massive corpus is passed to the task of implicit discourse relation recognition,which solves the problem of insufficient training datas and semantic ambiguity.Finally,the graph convolutional network is used to learn the graph representations according to the dependency relations and the node features in the static graph,so as to obtain the argument pair representation that integrates the syntactic information and semantic information of the arguments,which is beneficial to the inference of implicit discourse relation.The experimental results on the PDTB 2.0 dataset show that our model is comparable to some advanced models.(2)Implicit Discourse Relation Recognition based on Graph Attention NetworkStatic graph constructed based on dependency relations have shortcomings such as single semantic information,difficulty in correcting dependency parsing errors,and inability to jointly optimize with graph representation learning.Existing researchers use self-attention mechanism and bilinear model to learn dynamic graph,so that graph structure learning and graph representation learning can be jointly optimized end-to-end,but they ignore the sparse characteristic of argument interactive patterns.In response to these problems,we use the attention mechanism to mine the self-semantic information and interactive semantic information of the arguments,and embed them in the dynamic graph,and then use the graph attention network to learn the graph representations.Specifically,the dynamic graph structure is first learned based on the Prob Sparse Self-Attention mechanism,and the sparsification of argument interactive information is completed in the process.Then,the graph attention network is used to learn the graph representations of the discourse arguments,and the rich semantic information on the edges can be utilized by modifying the calculation method of the attention scores.Graph construction and graph representation can be jointly learned,and the graph structure can be continuously optimized in iterations to learn better graph representations.We conduct experiments on the PDTB 2.0 dataset.The Macro-averaged F1 of the four-class classifification can reach 65.28%,and the accuracy rate can reach 71.54%,which is better than the current advanced model.for the first time,we study the sensitivity of different attention mechanisms to the length of discourse argument pairs.The experimental results show that the Prob Sparse Self-Attention mechanism can improve the classification performance of the model for longer argument pairs.To sum up,we convert the original text sequence of discourse arguments into graph-structured data,integrate the self-semantic features of the arguments and their interactive patterns into the graph structure,and integrate the argument representations into the node embeddings.Two methods for implicit discourse relation recognition based on graph neural network are proposed.To a certain extent,this research provides an important reference for the related research of the discourse analysis,and have corresponding value in theoretical research and practical application.