Text Sentiment Classification Based On Attention Mechanism

With the increasing user penetration of the Internet,many people express their opinions on events or products on social media,e-commerce and other platforms.Therefore,a large amount of text data has appeared on the Internet.Sentiment classification,a key problem in sentiment analysis,can help consumers,businesses or governments to make proper choices or decisions,which has high commercial value and social significance.This thesis focuses on sentiment classification based on multi-task learning and aspect-level sentiment classification.Sentiment classification based on multi-task learning is to judge the overall sentiment of text.Models based on deep learning are prone to the overfitting problem when training data is less or limited.Multi-task learning is an effective strategy to improve the performance of a single task by sharing knowledge of related tasks.Existing sentiment classification work based on multi-task learning utilizes adversarial training to optimize features,aiming to avoid the mixture of features,or utilizes gate mechanism to control the flow of information between tasks.However,there lacks the importance measurement of features and irrelevant features will affect the judgment of sentiment.Aspect-level sentiment classification is to judge the sentiment of text given an aspect.Attention mechanism can be used to filter information and is widely applied to aspect-level sentiment classification.Existing work utilizes attention mechanism to obtain global features of text,but does not build the connection between local features of text and the aspect,which reduces the quality of local features.Besides,long shortterm memory network is commonly used as the feature extractor.However,it can not actually capture long-range dependencies or be calculated in parallel.Aiming at the problem of lack of the importance measurement of features in sentiment classification based on multi-task learning,the problem of lack of the relationshipbetween local features of text and the aspect,and the problem that long short-term memory network can not capture long-range dependencies or be calculated in parallel in aspect-level sentiment classification,main work of this thesis is as follows:(1)A sentiment classification method based on attention mechanism and multitask learning is proposed.Attention mechanism is introduced to the shared feature space and the private feature space,which assigns higher weights to important features and lower weights to irrelevant features.Besides,based on bidirectional long short-term memory networks,language modelling is introduced as an auxiliary task to enhance the ability of capturing syntactic and semantic information in the private module.In the end,two part features after attention are combined for sentiment classification.Compared with existing methods,the effectiveness of the proposed method is verified on 16 datasets of FDU-MTL.(2)An aspect-level sentiment classification method based on multi-granularity and multi-layer attention is proposed.Convolutional neural network is used to extract local features with different granularities of text.Then,multi-granularity attention is utilized to build the connection between the aspect and multiple local features,which improves the quality of local features after fusion.In order to capture correct contextual information,multi-layer attention is used to obtain the global representation of text for sentiment classification.Experiments are carried on Restaurant,Laptop and Twitter datasets and the results show that the proposed method achieves better performances than other benchmark models.(3)An aspect-level sentiment classification method based on multi-head self attention and fine-grained attention is proposed.Multi-head self attention directly models the relationship between words without distance limitation,and can be calculated in parallel.Use it as the feature extractor to efficiently extract multi-dimension semantic features of text and the aspect.On the basis of it,fine-grained attention is applied to distribute weights by building the connection between text and the aspect in word level,aiming to learn proper representations.Experiments are also carried on Restaurant,Laptop and Twitter datasets and the effectiveness of the proposed method is verified compared with other benchmark models.