Research On Sentiment Analysis Of Self-attention Mechanism Based On Pre-trained Language Model

With the prevalence of the Internet and the development of network technology,users can express their views on things on the network,and these views contain the emotional factors that users want to express.These comments with obvious emotions can directly reflect the views of users or netizens on this thing.With the advancement of big data technology and the improvement of hardware facilities,each platform will establish its own comment section,which makes the amount of information text data on the network platform Presenting a sharp growth trend,how to obtain effective information in the text more efficiently,and to more accurately explore the emotional factors in the comments has become an urgent requirement of the current industrial and academic circles.Sentiment analysis is also called opinion mining,subjective factor mining,etc.The process of solving a large number of online reviews and analyzing texts with emotional factors through computer analysis can effectively grasp the public's public opinion tendency on some things through information such as reviews.The main research contents of this article are as follows:(1)Aiming at the problem of insufficient classification effect of traditional sentiment classification models to accurately capture the relationship between words,this paper proposes a GE-Bi LSTM(Glove-ELMO-Bi LSTM)sentiment analysis based on pre-trained language model word vector fusion.The algorithm first trains the word vector with the pre-trained language model ELMO for the purpose of the language model,and then performs operation fusion with the training results of the traditional Glove model.It combines global information and local context information to increase the density of the word vector matrix.The characteristics between them are better expressed,and combined with Bi LSTM neural network can better capture the relationship of context information.The experimental results prove that the GE-Bi LSTM sentiment analysis algorithm can generate high-quality word vectors and test on the data set.The accuracy rate is 2.3% higher than the traditional model,and the F1 value is improved by 0.17.(2)Aiming at the problem that the traditional deep learning sentiment analysis model cannot accurately capture the relationship between semantics,this paper proposes a self-attention-based SABG(Self-Attention-Bi GRU)structure.This structure uses Self-Attention as the main Train the layer,obtain the keyword information,calculate the keyword weights through multi-position attention,and finally perform stitching.The output weight matrix is used as the new input.The residual network constructed by the multilayer Bi GRU(bidirectional GRU)is used to finally perform emotion classification.task.This structure can effectively prevent gradient dispersion,and experiments have proved that the effect is better than the traditional emotion classification model,and the accuracy rate is 1.4% higher than the traditional model.