Research On Graph Neural Network-based Multi-label Text Classification

With the widespread application of artificial intelligence and big data technology,retrieving useful information from colossal user data has become a topic of great concern for researchers and technicians.Text classification is the essential task of the information retrieval community,which automatically classifies unstructured text content on the Internet,such as news topics,user emotions,and so on.Nowadays,most text classification techniques have begun to use deep neural networks based on back propagation.Deep neural networks can capture more discriminative text features than traditional machine learning methods based on artificially constructed features.Among the existing deep text classification methods,one method based on graph neural networks has received widespread attention due to its superior performance.Specifically,a text classification method based on graph structure first converts text content into the form of a graph structure,then uses a specific graph neural network to encode features on the graph,and finally performs feature classification.This method can undoubtedly more effectively consider the dependencies between words in the text.However,by analyzing many text classification methods proposed by predecessors,this article believes these methods still have some problems.First,text classification tasks can be basically divided into single-label text classification tasks and multi-label text classification tasks,with the difference being whether the actual label of the text is one or more.Most of the former work will only consider text content information,ignoring the characteristics contained in the label itself.Although the related work of the latter takes into account the information of the label,they encode the text and label separately and finally splice and classify the features of the two.This simple method of feature stitching undoubtedly ignores the correlation between the text and the label;Secondly,existing text classification methods only use a supervised cross-entropy loss as their objective function to train depth models,but a single supervised objective function is not sufficient to make text features discriminative and robust.In order to solve the two problems mentioned above,this article proposes a new model called Text-Level Graph Networks with Contrastive Learning(TGNCL),which aims to solve the above problems by:(1)Combining the advantages of both,this article proposes a graph-based approach that not only constructs graphs for predefined labels but also constructs text graphs and merges the two into the same heterogeneous graph structure.This method not only considers the information contained in the label but also considers the correlation between the text and the label in the form of heterogeneous graphs.In addition,based on this heterogeneous graph,this paper also proposes a graph attention neural network based on moving averages,which enables the weights of edges in the graph neural network to be dynamically and reasonably updated.(2)In this paper,a graph contrastive learning regularization term is added to the original objective function,hoping that samples of different classes can be kept away,thereby making the features more discriminative.Moreover,graph comparison learning relies heavily on data enhancement of graph structures,and supervised learning of these enhanced graph data also achieves the goal of expanding training samples.Finally,this paper also verifies the accuracy of the TGNCL model proposed in this paper on four public datasets for multi-label and single-label text categorization tasks,such as AAPD and Reuters 21578 datasets.The experimental results show that the TGNCL model exceeds the baseline method compared to most data sets and experimental settings.In addition,sensitivity analysis and convergence analysis have been conducted to find the optimal combination of hyperparameters for the model,as well as the convergent iteration number.