Research On Text Classification Based On Deep Learning And Attention Mechanism

In recent years,deep learning methods have made important progress in text classification research.In contrast to traditional methods,deep learning based text classification methods use neural network models to train directly on text data,eliminating the need for manual feature extraction.However,there are problems such as single model selection,inability to model sentence hierarchy and insufficient text semantic feature vectors.To address these issues,this thesis investigates text classification by integrating deep learning and attention mechanisms.The main contributions are as follows.(1)To address the problems such as a single neural network unable to acquire local and contextual semantic features,and important textual information cannot be attended to in time.A multi-scale spatio-temporal hybrid text classification model based on a multi-head self-attention mechanism is proposed.The model improves the extraction of spatio-temporal hybrid features by acquiring dynamic word vectors through BERT and using a multi-head self-attention mechanism to focus on textual information that has a significant impact on the classification effect in a timely manner.Replacing traditional recurrent neural networks with ordered neuronal longand short-term memory networks to capture textual semantic information and long-range dependencies.Fusion of multi-channel convolutional neural networks and ordered neuronal long and short term memory networks to obtain spatio-temporal hybrid features which contain not only text local features of different granularity extracted from multi-scale convolutional kernels,but also text contextual semantic information and long distance dependencies.The results of experiments on the Taobao review dataset and the THUCNews dataset show that the model achieves 96.3%and 94.68%classification accuracy,and its recall and F1 values are better than those of the selected comparison models,verifying the superiority of the model.(2)To address the problems that the traditional model only uses word vectors as input in the text representation stage,which is prone to insufficient semantic feature vectors of the mined text,and that the one-way gated cyclic unit model has a single form of information transfer and cannot be transferred from two directions simultaneously.A hybrid neural network text classification model based on multi-features and attention mechanism is proposed.The model combines together word feature vectors,word feature vectors,lexical feature vectors and stroke feature vectors as inputs in a downstream task,relieve the problem of traditional models using only word vectors as inputs and insufficient extraction of semantic feature vectors.Inputting fused word vectors into a temporal convolutional network to extract features of the text in different time dimensions.Inputting the combined vectors into bidirectional gated recurrent unit and extracting feature information through two gated recurrent unit in opposite directions.Finally,an attention mechanism is introduced to allocate more attention to information that has a critical impact on the classification effect using a weight allocation mechanism.The results of the experiments on the SogouCS dataset and the FuDan news dataset show that the model outperforms the selected comparison model in terms of accuracy,recall and F1 value on both datasets,and its accuracy is 1.95%and 1.72%higher than that of the comparison model respectively,verifying the effectiveness of the model.