Research On Text Classification Model Based On Deep Learning And Attention Mechanism

With the continuous progress of Internet technology,the information on the Internet is more disorderly,especially the negative and false information in the mass of information,which has brought all kinds of adverse effects on society,so it is very necessary to sort out and summarize the information.Text classification technology has the advantages of fast and accurate automatic text classification,which has been paid great attention to and become a research hotspot.The early text classification methods mainly use the traditional machine learning algorithm,although there are some results,there are still a variety of problems.This paper mainly uses the deep learning method and attention mechanism to solve the problem of text classification.In order to improve the accuracy of classification,deep learning method is used to enrich the semantic features of text from the point of view of not rich semantic features;attention mechanism is used to increase the weight of the features that play an important role in the classification results from the point of view of not highlighting the information that plays an important role in the classification results.Therefore,the main work of this paper includes:(1)For the previous model,the state before and after the current moment cannot be considered at the same time,which leads to the situation that the final classification effect is not good.Besides,the traditional text classification model uses a single source word vector to directly delete or randomly initialize some words that do not appear,which causes the loss of semantic information and the single source word vector merges into a single channel,resulting in insufficient semantic features.Firstly,a CNN-based multi-channel feature representation text classification model(MC-CNN)is proposed.This method uses word vectors from different sources as the input of two bidirectional long and short-term memory networks,and the forward and reverse outputs at each moment are stacked vertically to form a multi-channel text feature representation,to capture the context information at the same time,and further enrich the semantic information of feature representation.Then the multi-scale convolutional network is used to further enable the model to fully consider the information before and after the current moment at the same time,to perform text classification more effectively.And the results on multiple datasets are compared with the previous model results to prove the effectiveness of the model.(2)Aiming at the shortcomings of the recurrent neural network that cannot extract the local key phrase information well when modeling the entire sequence,and the traditional text classification model cannot highlight the feature sequence that is important to the classification effect,an attention mechanism-based disconnected recurrent neural network text classification model is proposed(Att DRNN),this method limits the distance of recurrent neural network information transmission so that recurrent neural network can extract local position invariance like convolutional neural network and obtain the same representation of key phrase information.The attention mechanism is used to give more weight to important features to highlight the feature sequences that play an important role in the classification effect,thereby improving the classification accuracy.