Text Representation And Classification Based On Deep Learning With Improved Attention Mechanism

With the rapid development of the Internet and the advent of big data,various terminals generate massive amounts of text data.The rich semantic information contained in the text is of great value to the development of all walks of life in society.Text classification,as the basis and key technology of natural language processing,extracts a kind of coarse-grained text semantic information through preprocessing and classification of text data,which has a wide range of practical application prospects.In recent years,many network models in deep learning have been proven to have excellent performance in text feature representation and text classification.Based on the analysis and research of convolutional neural networks,recurrent neural networks,and attention mechanisms,the hybrid deep learning model combining attention mechanisms and neural networks to solve text representation and classification problems is in-depth researched.Some improvement methods are proposed.The main research contents are as follows:The definition and general process of text classification are introduced systematically,including text preprocessing,text representation,feature reduction,and common classification methods.Through the research on the common methods of each step and the analysis of the characteristics of text classification tasks,the defects of traditional text classification methods are further pointed out,which lays the foundation for the design of subsequent classification network models.Aiming at the problem that fixed convolution kernels in traditional convolutional neural networks can only extract fixed-size semantic features and cannot adapt to different text feature sizes,dense connections between convolutional layers are used instead of the traditional connection methods to adaptively extract multi-scale semantic features.Through the deep global-attention,the relationship between local features and the global feature expression of the text is strengthened,and make the model pay attention to the most important feature information of the entire input text.Experiments show that the model improves the quality of text feature representation and the effect of text classification.Aiming at the problem that the recurrent neural network is a biased model,the input of the sequence is more important than the effect of the previous input on the result,the gated recurrent network and attention mechanism are introduced to optimize the global representation of the sequence in the text representation.Based on the discussion of convolutional feature extraction methods,a multi-scale convolutional attention and weighted combination of hidden layer states of the gated recurrent network is proposed to obtain the attention distribution representation of semantic features,effectively highlighting the key in the sequence semantic information.Experimental results show that the model can improve the performance of input sequence feature representation and text classification.