Text Sentiment Analysis Based On Selfattention Mechanism And LSTM

With the vigorous development of the Internet information industry,people’s use of the Internet has generated a large amount of user raw data that is of great value to academia,industry,and commerce.To develop the value of these data,researchers focus on the study on the mining of emotional tendencies behind the raw data of users.The research method of deep learning based on artificial neural network has become the mainstream.Research that mines the emotional tendencies behind user raw data is called sentiment analysis.Researchers introduce neural networks into sentiment analysis research on text data.These networks are usually built based on LSTM sequence models and attention mechanisms.However,for LSTM family models,if the sequence is too long,the information of longdistance sequences will be lost,and gradient explosion is prone to occur.In addition,LSTM network-based models usually connect the forward and backward hidden state vectors directly,resulting in information redundancy at the current time step.Multi-head attention mechanism is a special structure of self-attention mechanism,which is widely used to capture relevant information between text sequences.Nevertheless,feature information is mapped and fused after subspace parallel computation,which makes each subspace information blurred and cannot be fully utilized in downstream tasks.Furthermore,too long text sequence will increase the computational complexity of the multi-head attention mechanism module.Based on the above analysis,this paper proposes a new sentence-level sentiment analysis model IBMS-LSTM.We divide the sequence into different information blocks,which are fed into the model.The multi-head attention mechanism is used to extract key features of the words block and capture the relationship between the words in each information.The information features of different subspaces in each information block interact with each other via constructing a multi-space bidirectional LSTM.The above design not only alleviates the gradient explosion problem caused by long-sequence text to the LSTM module,but also reduces the computational complexity of the multi-head attention mechanism for processing long-sequence text.This paper also proposes a dual fusion mechanism,which is used to fuse sentence representations in each subspace and mine the latent emotional semantics,reducing information redundancy,and obtain the final representation of sentences for sentiment classification.This paper builds detailed experiments and uses multiple public datasets to demonstrate the effectiveness of the IBMS-LSTM model.