Deep Graph Neural Networks For Text Classification

As we move into the information age,we are surrounded by a large amount of texts in the real world.It is a challenging problem to process these texts effectively and extract the information we need.Benefit from the rapid development in text mining,text classification has attracted a lot of attention from both academia and industry and has made significant progress.Text classification is a very classical and fundamental task in natural language processing,which aims to assign the correct label to a given text and has been widely used in various practical applications,such as question answering,topic classification,and paper publication venue prediction.Compared with earlier rule-based text classification methods,statistical-based text classification methods have better accuracy and stability.However,they still rely heavily on manual feature engineering,a process that requires rigorous processing or specialized domain knowledge and can be time-consuming and costly.At the same time,these models do not take full advantage of the extensive training data,as the relevant features are already defined in advance.Recently,the emergence of deep learning has dramatically changed the field of artificial intelligence.These deep learning methods can automatically model complex features and produce semantic and contextual representations of text,thus eliminating the tedious and complex manual feature design process.It has become the dominant paradigm for natural language processing tasks,including text classification.Although previous sequential-based deep learning models can capture semantic and syntactic information in local sequences of consecutive words well and have made impressive progress in text classification tasks,they still have some limitations.First,they do not capture long-distance word interactions well,thus ignoring the global cooccurrence information of these words.Second,they ignore the grammatical or syntactic structure within the text,which is helpful for correctly understanding the text.Recently,graph neural networks(GNNs)are becoming a research hotspot due to their powerful performance in handling complex structural data and relations.A series of GNN-based models have achieved impressive performance in text classification tasks.This work first investigates a specific application scenario of text classification,i.e.,paper publication venue prediction.For this application scenario,the previously proposed models ignore the structural information inside the papers,using handcrafted features to represent the papers while missing those features that involve highlevel semantics.To address the above issues,this work proposes constructing semantic graphs for each paper abstract and executing a dual attention message passing neural network to obtain their discriminative paper abstract representations.Extensive experimental results on relevant datasets show that the performance of the proposed model is excellent,consistently outperforming existing baseline approaches.Next,this work analyzes the drawbacks of previously proposed GNN-based text classification models,i.e.,they only consider the one-hop neighbors of words within the text and suffer from oversmoothing problems if many GNN layers are stacked.To address these limitations,this work proposes a deep graph attention diffusion model for text classification tasks.Specifically,the model first uses attentional diffusion techniques to widen the receptive field of words in the text,which can capture longrange word interactions at each layer.In addition,to train a deeper network to extract the hidden semantics of words,the model decouples the feature transformation and feature propagation processes of GNNs,which can alleviate the oversmoothing problem.The performance on a series of benchmark datasets demonstrates the superiority of the model proposed in this work.