Research On Aspect-Based Sentiment Analysis Based On Attention Networks And Affective Word Embedding

Sentiment analysis is an important task in natural language processing.It obtains users' sentimental opinion tendency by mining users' comments and has great practical significance.Sentiment analysis can be divided into document-based sentiment analysis,sentence-based sentiment analysis and aspect-based sentiment analysis according to the levels of granularity.Among them,aspect-based sentiment analysis is the most special,which needs to analyze the polarity of sentiment for different targets in the comment text.Compared with the other two kinds of sentiment analysis,aspect-based sentiment analysis is directed,so it has more value and has attracted more and more researchers' attention in recent years.Deep learning is a powerful machine learning model and has gradually become the mainstream method of aspect-based sentiment analysis.Especially attention networks which use attention mechanism have achieved great success in recent years.But the target text information is not fully utilized in existing models.Because sentiment analysis has been considered as a special text classification task,some researchers regard the comment text as a whole and ignore the importance of the target text.Though some researchers consider the target as independent information,they do not calculate the attention at the target level.The lack of affective information in word embedding also affects the accuracy of model classification.To solve the above problems,this paper mainly focuses on attention networks and uses both context and target to study aspect-based sentiment analysis.The main contributions are outlined as follows:1.To make better use of target,we build the multi-head attention over attention model(MHAOA).We use the idea of attention over attention in question answering system to construct the attention network at first.Though this model can make use of both target information and context information,it uses dot product to calculate the attention score which can't build the relationship between target and context fully.Therefore,we use a new calculation method to calculate attention score instead of dot product,add location information to attention model and use multi-head attention to enhance the robustness of the attention model.We name the attention model multi-head attention over attention because it has the characteristics of attention over attention and multi-head attention at the same time.Due to the difference of the basic network,the performance of the model will vary greatly.In order to more accurately verify the validity and applicability of MHAOA in aspect-based sentiment analysis,we use the attention model in three different kinds of based network to get three kinds of neural networks based on MHAOA and we named them MHAOA-Glo Ve,which is based on word vector encode network,MHAOA-LSTM,which is based on LSTM network and MHAOA-BERT,which is based on BERT network.Then we compare the three models with other existing models on three public datasets by the two metric,accuracy and macro average F1.The experiment results show that the three models in this paper have competitive performance so it proves that the multi-head attention over attention model can effectively match the corresponding context description to the target,so as to improve the final sentimental classification results of the models.2.To solve the problem that the word embedding in MHAOA lacks affective information,we construct affective word embedding.In the further analysis of MHAOA,we find that the lack of affective information of the input word embedding affects the performance of the model's sentimental analysis.In order to further improve the model of this paper,we add the commonsense information of the affective knowledge into the distributed word embedding as a kind of prior knowledge,and the affective word embedding with affective information is generated as the input.We compare the affective word embedding with the original distributed word embedding on MHAOA-Glo Ve and MHAOA-LSTM.The experiment results show that the affective word embedding is helpful to the better classification of sentiment.