Sentiment Analysis Based On A Two-level Attention Mechanism Network Fused With Emoticons

As the focus of social media has shifted from the PC terminals to mobile terminals,We Media has also entered a new stage.More and more people have become content creators,and people increasingly used to express their opinions and mood on some social media platforms such as Weibo.Data mining or sentiment analysis on these contents posted by users not only helps enterprises to make decisions,but also the government to carry out public opinion management and policy formulation,which has significant commercial value and social significance.As an important branch of natural language processing field,sentiment analysis has been studied by a large number of scholars at home and abroad.In addition to researchers in computer science,it has also attracted a vast of interdisciplinary researchers in sociology and computer science.The traditional method of sentiment analysis can be roughly classified into the lexicon-based or rule-based method and the machine learning method.The former requires a lot of human resources,besides,due to the cyberlanguage evolves extremely fast,which brings difficulties to the maintenance of the lexicon.All in all,this method exists some limitations,and generally plays the role of supplementary means in practical applications.The latter heavily relies on feature engineering,the eneralization ability of trained model is poor and cannot meet the needs of cross-domain sentiment analysis.In recent years,the deep learning method has developed rapidly,it can automatically learn features.At the same time,some networks are very suitable for text analysis for the superiority of its structure.Taking Weibo and Twitter for example,most of the current deep learning methods always only focus on textual information,while ignoring other media information(such as emoticons)containing important emotional cues.And those methods that notice this kind of information do not effectively solve the emotion ambiguity problem of emoji,and do not study deeply the influence of emoji on the emotion expression of the text.In order to complete the shortages of past research methods,we propose an emoji representation model that integrates multi-dimensional information.We also build a sentiment analysis model based on a two-layer attention mechanism network,and exhibit hypothesis proof of related theories.The main research of this paper includes:(1)This paper proposes an emoji representation model that integrates multidimensional information.Firstly,the hypothesis test is conducted to demonstrate that for a certain user,his behavior of using emoji is relatively fixed,i.e.,for a certain user,the emotion polarity of emoji is relatively fixed.Then we use co-occurrence information and word embedding to get the emotion polarity of non-ambiguous emoji and ambiguous emoji,thus obtain the emotion information of emojis.In the end,we get the emoji representation by combining the position information,semantic information,emotion information and occurrence frequency information of emoji.(2)We also propose a sentiment analysis model based on a two-layer attention mechanism network.We obtain clause vectors in the word-level attention layer,then in the clause-level attention layer,we combined the clause vectors with emoji representation vectors to explore the emotional impact of emoji to different clauses in the document,finally,we get the document vector.We put the document vector into the sentiment classification model to get the result of binary classification.(3)Experiments on two real-world datasets demonstrate that the emoji representation method of using multi-dimensional information is superior to the traditional representation method which using only the emotion information,it also manifests that user identity information can indeed eliminate to some extent the emotion ambiguity of emoji.At the same time,the model that integrates attention mechanism with emoji representation is better than other existing deep learning models,which illustrates the effectiveness of the emoji representation method proposed in this paper,and shows that emoji have some effects on the sentiment to clauses,indicates that the theoretical of the clause-level attention mechanism is reasonable.