Research On Text Classification Model Based On BGRU And Self-Attention Mechanism

Big data interconnection has led to the explosive growth of electronic data.As one of the most important information carriers,text has become the most common form of data display.Text classification is involved in many fields such as artificial intelligence and pattern recognition.It has important academic research significance and commercial value.How to make the computer accurately locate the effective information in the text and realize automatic classification has become one of the research hotspots.With the diversification of text structure and complexity of content,shallow machine learning methods can no longer meet people's classification needs.Deep models can automatically learn semantic structure and potential information to generate deeper text representations to avoid complex feature engineering.At present,commonly used methods in the field include Convolutional Neural Network(CNN),Recurrent Neural Network(RNN)variants and attention mechanism.Their application makes the feature expression of text better than previous machine learning methods.But there is still a lot of room for improvement in terms of model accuracy,robustness,and convergence speed.This thesis focuses on deep learning models that are often used to process text classification tasks and perform well,and improve the classification performance of the model by improving the existing baseline algorithms.The main innovations and work results of this thesis are as follows:(1)Aiming at the defect that CNN is good at capturing local detailed features but cannot pay attention to the long-distance dependence of long-sequence data,and RNN can model long-sequence texts,but because of the long-term iterative operation,the local key information is easily ignored.This thesis introduces a new type of network structure D-BGRU by introducing the information interruption mechanism into the BGRU.The structure retains the original advantages of BGRU while adding position invariance similar to convolution characteristics,which realizes the combination of timing characteristics and spatial characteristics.The results of simulation experiments show that DBGRU has a certain improvement in the classification index compared with the existing baseline model.The results of simulation experiments show that D-BGRU has a certain improvement in the classification index compared with the existing baseline model.(2)On the basis of the D-BGRU model,the self-attention mechanism is further introduced to generate the D-BGRU-SA model,which focuses on important feature information through the selfattention mechanism,and takes into account the impact of important information on the classification results,so as to assign higher key features The weight of further realizes the improvement of text feature engineering.The validity of the D-BGRU-SA model is verified by comparison with D-BGRU and other deep models.(3)In order to realize the multi-angle feature extraction of the text,we try to introduce the emotion dictionary method into the original model.Emotional features can be generated by convolution operation after filtering out the words containing emotional polarity in the original text through the emotional dictionary.The emotional features generated by the convolution channel are fused with the text features extracted by the D-BGRU model to achieve multi-angle optimization of the original text,and finally the generated feature vectors containing various information are performed attention operations.Through comparative experiments on multiple sentiment classification data sets with other sentiment classification model methods,the results show that the dual-channel model classification method fused with sentiment features proposed in this paper has achieved the improvement of classification accuracy,thus verifying the effectiveness of the model.