Research On Sentiment Classification Based On Deep Learning And Self-Attention Mechanism

Traditional emotion classification algorithms are mo stly based on shallow machine learning,and feature extraction is carried out by using manually designed feature selection method.However,these methods are time consuming,difficult to train,and high labor cost defects are difficult to apply to the huge application scenarios of today's data sets.The sentiment classification method based on deep learning can actively learn the word vector containing semantic information from the massive data,and obtain the feature and emotional expression of the sentence or document through different deep neural networks.Loss function in deep neural network has significant influence on overfitting of model training,and optimization of loss function can improve model generalization ability and reduce overfitting;Emotional words occupy an important position in text classification.The circulation neural network quickly sorts the contribution of input words in sentiment classification results,increases the influence of emotional words in text classification,and can quantitatively reduce the loss of emotional information;Introducing a self-attention mechanism in the emotional classification task,can fully learn the word dependence within the sentence,effectively solve the information redundancy and optimize the feature vector.Based on the above ideas,this paper comb:ines deep neural network and self-attention mechanism to develop a text sentiment classification method.By designing model structure and optimizing strategy,four kinds of emotion classification models are put forward in order to get better classification efifect.The main research work and innovation points of this paper are as follows:(1)Based on long-term and short-term memory networks and convolutional neural networks,the cross-entropy loss function used in the binary classification task is improved,so that the model can more effectively fit the prediction error samples and reduce the over-fitting.Based on the optimization of the cross entropy loss function,design the LSTM-BO(Long-Short-Term Memory Binary-Optimize)and CNN-BO(Convolutional Neural Networks Binary-Optimize)model,and in Chinese and English two kinds of data sets parameters optimization experiment and comparative analysis.Experiments show that the LSTM-BO and CNN-BO models can improve the accuracy of sentiment classification to a certain extent,significantly reduce the loss rate and prevent over-fitting.(2)The circulation neural network is capable of processing sequence information of text data,by calculating the degree of influence of the input words on the final classification results and sorting them.According to the ranking results,words with strong emotional tendency are given higher weight to reduce the loss of emotional information.Based on this,this paper has designed a W-RNN(Weight-Recurrent Neural Network)model,which has been proved to be more effective through quantitative and qualitative experiments in both Chinese and English data sets.(3)The attention mechanism can help the algorithm model to discover the key features,and the self-attention mechanism can more effectively capture the internal structure of the sentence and optimize the feature vector.This paper proposes a strategy to solve the emotion classification problem by combining self-attention mechanism with bidirectional long and short term memory network.Experiments verify that the SA-BiLSTM(Self Attention-BiLSTM)model integrated with the Attention mechanism is easier to capture the features of the long distance interdependence in sentences,and can effectively solve the problem of information redundancy,thus further improving the accuracy of emotion classification.