Deep Learning Based Sentiment Word Vector Towards Sentiment Analysis

Sentiment Analysis (SA) aims at identifying sentiment polarity or strength of given text or certain fragments (e.g., sentences, phrases or words), which has wide applicants. For example, sentiment analysis in products reviews is able to identify the users’ sentiment orientation towards given products, which is helpful for merchants or consumers decision. Most previous work adopted machine learning approaches with the aid of abundant manually-designed features, which cost a lot of expert knowledge and time. Recently, researchers focus on extracting features automatically by using deep learning methods. As an important achievement of deep learning in NLP, word vectors (i.e., distributed word representation) are learned on the basic of contextual information (semantic and syntactic) and have been applied in many NLP tasks. However, due to lack of sentiment information, these learned word vectors are not effective to settle SA tasks as expected.To address this sentiment shortage issue, the purpose of our first work is to integrate sentiment information into word representation. Thus we present two sentiment word vector learning frameworks based on Google Skip-gram and a Convolutional Neural Network respectively. In each framework, we propose three models according to different strategies of information integration. To evaluate the effectiveness of proposed models, we conducted a series of qualitative and quantitative experiments in different languages and domains. This work has been published in International Conference on Asian Language Processing 2015 (IALP 2015) and International Joint Conference on Neural Network 2016 (IJCNN 2016).To expand sentiment word vectors from words to text, previous work directly concatenated word vectors of each word in text into a text representation, which neglects the word sequence information. To address it, our second work is to adopt a Convolutional Neural Network model to perform text modeling. Experimental results indicated the effectiveness of CNN for text modeling. This research has been applied to the SemEval 2015 and 2016 tasks and achieved good results. This work has been published in SemEval conference of 2015 and 2016.Besides sentiment orientation classification, our third work is to perform the sentiment strength prediction using the learned sentiment word vectors. To address it, we combined the learned sentiment word vectors and traditional manual features, and then adopted a pair-wise learning-to-rank algorithm to perform sentiment strength. Recently, this work achieved the 1st rank in 2016 SemEval Task 7, i.e., Determining Sentiment Intensity of English Phrases and will be published in 2016 SemEval.We conducted a series of experiments to preform sentiment word vectors learning and then examined the performance in sentiment polarity classification and strength prediction on different text levels (i.e., word-, phrase- and sentence-level), languages (i.e., English and Chinese) and domains (i.e., tweet and movie review). The experimental results show that our proposed models for learning sentiment word vectors have good generalization and are effective.