Research On Multi-level Implicit Discourse Relation Recognition

As a quite challenging task in the natural language processing,implicit discourse relation recognition aim to automatically judge the semantic relationship(contrast or cause)between two connective-absent arguments,i.e.clauses or sentences.In recent years,with the extensive application of deep learning in various tasks of natural language processing,various deep-learning-based models are proposed to implicit discourse relation recognition and have achieved advanced results while compared to traditional methods of artificial feature extraction.The existing deep-learning-based methods can be roughly classified into the following three categories: 1)Argument encoding based methods.Firstly,the neural network model is applied to learn the semantic representation of two arguments.Second,the semantic relationship is inferred based on the learned results.2)Argument interaction-based methods.During these methods,various interactive ways are applied to modeling the semantic relationship between words or phrases explicitly.3)Methods with explicit discourse data.These methods try to integrate a large number of natural labeled explicit discourse data to solve the problem of insufficient training data.In summary,these existing models mainly study how to learn the semantic representation of two arguments and model the relationship between them.Furthermore,the classification model was built for each level of discourse relationship.Inevitably,the semantic relationship between the multilevels of discourse relationship was ignored.In this paper,we mainly study how to use the semantic level characteristics of multi-level discourse senses and their mapping relations to improve the performance of multi-level discourse relations recognition.The main research contents are as follows:1)Hierarchical multi-task learning with CRF for implicit discourse relation recognition.During this model,multilevel discourse senses are used as supervision signals in different neural network layers.In this way,the feature layers are hierarchically shared.As a result,the hierarchical semantics of multi-level senses are explicitly encoded into features at different layers.To further exploit the mapping relationships between senses at adjacent levels,a Conditional Random Fields(CRF)layer is introduced to perform collective sense predictions.The experimental results showed that this method can significantly achieve better and more consistent results over competitive baselines on multi-level implicit discourse relation recognition.2)Sequence generation model for multi-level implicit discourse relationship recognition.On the one hand,the above-mentioned hierarchical sharing mode lack of flexibility causes the low-level discourse relationship to share all the features of the high-level discourse relationship.On the other hand,the above-mentioned CRF layer can only model the mapping between adjacent level discourse relations.Any two levels of discourse relations may exist dependence.Based on these considerations,we propose a sequence generation model for multi-level discourse relationship recognition.Specifically,the shared semantic features selected based on the gating mechanism,the semantic features specific to the current level extracted through attention mechanism and the prediction results of the last discourse relationship are combined to predict the current discourse relationship.By analogy,a multi-level discourse relation sequence can be generated.The model can not only realize information sharing among multilevel discourse relations flexibly but also model their mapping conveniently.The experimental results showed that the model has better recognition performance than the above model based on hierarchical multi-task learning and CRF.