Research On Short Text Classification Based On Feature Representation And Graph Convolutional Network

With the rapid development of the mobile internet,countless short text data are generated on the internet every day,and they will widely exist in various forms such as news headlines,social media information,and short messages.Short text classification is a basic task in natural language processing.It can be used in many follow-up tasks,such as public opinion control,sentiment analysis,news classification,etc.These short text data contain very important economic and academic value.Recently,the rapid development of graph convolutional network technology has attracted widespread attention,and it has been successfully applied in various fields,such as network analysis and natural language processing.Graph convolutional network is a simple and efficient graph neural network.It can well capture the high-order neighborhood information between nodes.We model text data as graph nodes,which can effectively capture the global vocabulary information in the text,which has made great progress in text classification.However,the graph convolutional network cannot well obtain the local context information of the text,such as the word order.On the other hand,due to the short text sentence,there are problems such as semantic sparsity and ambiguity at the same time.In order to solve the above problems,this thesis starts from the two aspects of graph convolutional network classification model and text feature representation,and conducts the following research work:(1)This thesis proposes a BTM graph convolutional network model(BTM-GCN)for short text classification.After the short text data is preprocessed,the BTM topic model is used for topic training,and then the training results are compared with Short text performs text modeling based on heterogeneous graphs.The heterogeneous graph flexibly models the semantic information between words,documents,and subject objects,and at the same time incorporates potential semantic related information,and uses graph convolutional networks to capture the semantic information embedded in the neighborhood node of the document node alleviates the problem of semantic sparsity of short text.This thesis conducted experiments on multiple data sets and achieved better classification performance than the benchmark model,indicating that the BTM-GCN model we proposed is a very effective classification model.Finally,the influence of each component of the model on the final classification result is analyzed,and the influence of parameters such as the number of topics,the number of the most relevant topics,and the size of the sliding window on the model effect is explored.(2)GCN that only considers global vocabulary information may not well capture the local context information of the text,such as word order,which is very important for understanding the meaning of sentences.This thesis proposes the BERT-GCN model,which integrates the vocabulary graph embedding module with BERT.The goal is to supplement the local semantic information captured by BERT with global information on the vocabulary,and allow these two types of information to pass through multiple layers attention mechanism for interactive fusion.We performed classification experiments on three short text data sets.The results show that the graph embedding obtained by graph convolution does provide useful global information for BERT,thereby improving performance.Compared with only using GCN and BERT,our model can achieve significantly better classification results,which demonstrates the effectiveness of BERT-GCN for interactive fusion of these two types of semantic feature representations.