Research On Text Classification Method Based On Deep Learning And Attention Mechanism

With the rapid development of computer information technology and mobile Internet,a large amount of text data is generated every day.Faced with massive data,it is necessary to efficiently obtain valuable information.In order to meet people's personalized needs for information,corresponding processing technologies need to be used to process and process massive text data,and text classification technology is the cornerstone of these technologies.Text classification is a classic topic in the field of natural language processing.Traditional text classification methods usually use shallow machine learning algorithms to extract features through artificially designed feature selection methods.Such methods have high labor costs,long time,and difficult training,and have poor adaptability to large data processing scenarios.The text classification method based on deep learning can automatically perform feature learning and feature extraction from massive text data,which greatly reduces labor costs and is easy to train,and enhances the mobility of algorithm fields.Word vectors,as a special text representation,can represent words with similar semantics and avoid the semantic gap existing in traditional methods.Using the self-attention mechanism in the text classification task can fully learn the text features,find important features,ignore secondary features,and capture key information in the text.Based on the above characteristics,this paper combines deep learning and selfattention mechanisms to study different types of text classification tasks.The main research work is as follows:(1)Using the word embedding mechanism to solve the high-dimensional,semantic gap problem of data representation in traditional text classification models.The word embedding mechanism maps text data to low-dimensional real number vectors to avoid dimensional disaster caused by high-dimensional input.Word vector synonyms trained using this mechanism have similar characteristics,so that the vector representation of words has certain basic semantic information and is effective.Avoid the semantic divide.In this paper,for data sets in different fields,the Word2 vec framework is used to pre-train the word vectors.(2)Aiming at the task of single-label text classification,this paper proposes a DSA-CNN model based on self-attention mechanism for single-label text classification.DSA-CNN uses convolutional neural network convolution and pooling structure to further extract hidden semantic features in text.DSA-CNN integrates the self-attention mechanism at the input layer and the pooling layer to capture the internal structure and dependencies of the text.It gives higher weight to important feature words,which can effectively reduce information redundancy and information loss when extracting feature vectors.It can highlight the role of keywords.This paper proves the effectiveness of the model by designing experiments on multiple data sets.(3)Aiming at the task of multi-label text classification,this paper proposes a SA-GRU model based on self-attention mechanism for multi-label text classification.SA-GRU combines GRU network to model text information,captures long context information in text sequence,and uses self-attention mechanism to score and assign weights to hidden layer units of GRU network.SA-GRU considers the semantic correlation between tags,and scores words in sentences according to tag semantics.Finally,it integrates attention score and tag semantic score.This mechanism can associate tag semantics with the words that the tag is concerned to,and to some extent alleviate the tail tags.This makes training difficult due to uneven distribution of label data.This paper validates the effectiveness of SA-GRU by designing experiments on the "Toxic Comment" data set.