| With the continuous advancement of information technology,the digital information era faces a serious challenge of processing massive data.Efficiently applying text classification technology has important practical significance for achieving rapid social development.Using deep learning methods to process text classification tasks can handle complex text features and achieve significant classification performance.In particular,graph neural networks can effectively capture the semantic and structural information between textual data as they can handle graphstructured data,leading to great performance improvements when applied to text classification tasks.Graph neural networks have shown significant performance in handling classification tasks,but existing graph-based classification models still have room for improvement.Firstly,the models tend to ignore the sensitivity of key nodes in the text graph to the predicted results.During message propagation,some key nodes in the text graph have a significant impact on the predicted result.Some models train all nodes with the same importance,which reduces the sensitivity of the model in the feature extraction process.Secondly,node feature vector representations are difficult to aggregate and update with global structural information.When constructing a text graph-based classification model,the graph nodes overly rely on local structural information during message propagation,ignoring the correlations between distant data,which makes it difficult to combine global structural information to update node vector representations.Currently,the GAT network uses the Attention mechanism to help the model more accurately focus on the relationships between different nodes,and the multiple layers of convolution operations used in GCN increase the perception ability of global structural information.Based on the ideas of the GAT and GCN models in addressing the above two issues,this paper proposes a text classification method,BGNN-A,based on BERT pre-training model and GNN model.The main content and contributions of this paper are as follows:(1)The BERT pre-training dataset is used in this study.During the text graph construction process,individual documents are transformed into graphs,with nodes being composed of the words in the document.The text dataset is preprocessed,and the words in the data are tokenized.The BERT pre-training model is then used to encode the words,and the encoded word vectors are used as the vector representations of the nodes in the text graph.Edges are then constructed based on the co-occurrence relationships between the words,thus completing the construction of the text graph.(2)A GGNN-A model is proposed based on the GGNN model,which incorporates attention units on top of the GRU units in GGNN.The attention unit can determine the degree of association between a node and its neighboring nodes based on its feature information,such as part-of-speech tagging,and weight the relationships between nodes.It can also learn the information transfer between nodes.During the message passing process,the vector representations of nodes and their neighboring node feature information are continuously aggregated and updated,fully utilizing global structural information and long-range dependency relationships.The text graph is trained using the GGNN-A and GCN models,and the resulting node vector representations are concatenated and fused,and then pooled.(3)The combined method of self-attention pooling and mean-max pooling was used for dimensionality reduction.Self-attention pooling was used to weight the node vectors with attention weights so that the model could focus on more important vector representations.Mean-max pooling was used to combine both the most important feature information in the node vectors and the feature information in the entire node vector set during the dimensionality reduction process.Finally,the two pooling methods were combined to form the final graph representation vector,which was input into the classifier to predict the corresponding classification label.The combination of the two pooling methods significantly improved the model’s classification accuracy and robustness.This article conducted experiments on the newly proposed classification method BGNN-A,and compared its performance with benchmark models.The results showed that the BGNN-A model has superior classification performance. |
