Text Classification Research Based On Deep Neural Network And Attention Mechanism

With the booming Internet,information in the form of text has presented explosive growth.Text information contains a lot of rich information,but it is unstructured.How to effectively manage the text information and mine the valuable information is still the challenge of modern era.As one of the important tasks of natural language processing,text classification has been gradually applied to many fields.The key of text classification is the vectorization of text and sentence modeling.Traditional text feature vector representation with the defects of high dimensionality and high sparsity ignores the semantic relevance of context.With the excellent performance of deep learning in image processing,speech recognition and other fields,it has also been proved to have the ability to extract higher-level representations of sentences or texts in many complex natural language processing tasks.However,a single deep learning model still has the problem of ignoring contextual semantic relations or not distinguish key information.Therefore,combining the advantages of each deep learning model,extracting more semantic and distinguishing features with higher level has become a research hotspot..On the basis of summarizing and studying deep learning model and text classification technology,this paper makes an in-depth study on on how to optimize the deep learning model based on attention mechanism to solve the text classification problem.The main research work of this paper is:(1)This paper studies the key steps in the text classification process and the existing existing classification models.After research and analysis,the deep neural network has good learning and expression ability in text features,and the deep neural network model can be used for text classification tasks.For the variant of the network model,Bi-directional Long and Short Memory Neural Network(BiLSTM),the semantic features of the text are considered from the context information without distinguishing the contribution of the context information on the text features.In this paper,a contribution rate is firstly introduced based on the bidirectional memory neural network(BiLSTM),and the BiLSTM model with the contribution rate is designed and implemented.The improved model can adjust the contribution of historical information and future information to semantics,and further improve the text classification efficiency.(2)For the lack of semantic for single model and key feature selection problems,in order to highlight the influence of the key features and reduce the loss of important information,in this paper,the attention mechanism is used to combine the convolution neural network(CNN)and the BiLSTM model to propose a new model for text classification.The model applies the CNN and the improved BiLSTM to obtain a series of advanced local convolution features and the intermediate vector representation of sentences respectively,and calculates the weight value of attention through the attention mechanism to obtain the final text feature representation.The model integrated with attention mechanism not only retains effective information,but also solves the problem of information redundancy and key text information loss.The classification performance and accuracy are further improved by optimizing the text feature vector.