BiLSTM-CNN Text Classification Based On Attention Mechanism And Residual Connection

The rapid development of Internet and mobile Internet applications has led to the explosive growth of text data,and it has become impossible to manually sort and organize text.How to implement rapid classification of text in massive text data and apply it to subsequent article recommendation,semantic analysis,information retrieval,information extraction and machine translation has always been a hot topic in the industry.With the continuous development of deep learning technology,more and more scholars have applied deep learning technology to the field of natural language,and have achieved very good results.However,existing algorithms often fail to accurately express text information and network degradation problems in deep networks.Based on the analysis of existing text classification algorithms,the main research work is as follows:This paper is difficult to converge in training for the use of high-dimensional text word vectors in text categorization.The pre-processed text corpus is pre-trained using the word2 vec algorithm.Using the obtained vectorized text data as the input of the entire classification model,the dimensional disaster is avoided and the convergence of the model is accelerated.In this paper,in the text classification algorithm based on deep learning technology,the convolutional neural network(CNN)can not obtain the global features of the text,and the bidirectional cyclic neural network(Bi LSTM)can not focus on the local features of the text.This paper combines CNN and Bi LSTM to extract text.When the feature information is used,the local features of the text can be extracted through the CNN network,and the global features of the text can be extracted through the Bi LSTM network,which solves the problem of feature extraction in text classification.In this paper,the influence of different words on the text classification results in the text is different.This paper introduces the attention mechanism in the model,calculates the probability distribution of the input information on the classification result,and the input text through the probability distribution.The feature vector is optimized to obtain key word features that affect the text classification result.In this paper,when the number of layers of the neural network model is too deep,the residual problem of the neural network is introduced.It ensures that deep network parameters can be updated and learned in deep networks.Finally,in order to verify the validity of the model and algorithm,this paper designs and implements a comparative experiment.The experimental results show that the accuracy,precision and recall rate of the proposed model in the four corpora have achieved the expected results.