Research On News Text Classification Based On Multi-head Attention Mechanism And Feature Fusion

The rapid development of today's Internet technology is accompanied by the generation of massive and disorderly network information.Therefore,it is necessary to adopt effective information management methods.News text is an important carrier of information.How to classify massive news texts to dig out valuable information has become one of the current hot research topics.Facing the task of Chinese news text classification,traditional text representation methods cannot solve the problems of polysemous word and cannot provide rich contextual semantic representation for the text.Embedding representation is obtained by text representation method.In the process of text feature extraction,there is no global semantic dependency between text features,and the amount of text information contained in the final classification features obtained is not comprehensive enough,and the granularity and diversification of features is lacking.This paper proposes a news text classification model based on multi-head attention mechanism and feature fusion.In view of the limitations of traditional text representation methods,the BERT pre-training language model is used as the method of training word vectors.This method is based on the advanced two-way Transformer encoder to extract the semantic features of the text.The text representation generated by this dynamic representation method includes A wealth of text position information and sequence information.In order to obtain the local key information of the text with multi-faceted semantics,the Multi-Head Attention mechanism is incorporated in the process of extracting the local features of the text.First,the character-level granularity Embedding obtained by the BERT model of the text is input into the convolutional neural network,and multiple convolution kernels of three sizes are used to extract word fragment information at different distances,and representative local text semantic features are output after pooling and dropout operations,and then a multi-head attention mechanism is used to extract different semantic representations.Capturing text information in the space,enhancing the dependency between local features,and generating local key information with multiple semantics.In order to maximize the amount of information that the news text finally represents,from the two perspectives of multi-granularity of text features and the combination of the structural characteristics of news content,the following three parts of text features are merged as the final classification features of news text: The character-level representation of the full-text content output by the BERT model is input into the combined module of the convolutional neural network and the multi-head attention mechanism to obtain the local key features of the text in various semantics;the second is the sentence-level representation of the full-text content output by the BERT model;The third is to retain the important information in the news text to a greater extent,extract the title part separately,and output the sentence-level representation of the title content through the BERT model.Finally,the final classification features of the text generated after the fusion are input into the Softmax classifier to complete the final classification task.Based on the THUNews news text data set published by the NLP Group of Tsinghua University,experiments are carried out,and the classification results obtained by the model proposed in this paper are compared with the benchmark model,and ablation analysis is performed.The experimental results show that the classification results of the model proposed in this paper are better than the benchmark model in all indicators.The average accuracy,recall,and F1 value reach 96.54%,96.51%,and 96.50%,respectively,and have achieved the expected results in the ablation analysis.As a result,the validity and rationality of the model was verified.