The Study Of Sentiment Commonsense Induced Neural Networks For Sentiment Classification

Sentiment classification,aiming at classifying a given sentence into a certain sentiment polarity(positive or negative),is one of fundamental tasks in Sentiment Analysis(SA).Since the Neural Network(NN)has ingenious model structure and remarkable representation ability,it has achieved state-of-the-art performance on many sentiment classification tasks.However,when the neural network utilizes massive training data to modeling,its complicated Black Box learning process is not aware of the existing sentiment commonsense,which plays a decisive role in sentiment classification task.For example,the sentiment polarity of the sentiment words explicitly determines the sentiment of the sentence in most cases.Therefore,it is worth exploring how to integrate sentiment commonsense as a prior knowledge into neural networks to improve models' performance and interpretability.The major work of this paper are as follows:(1)First,this paper utilizes Attention Mechanism to integrate sentiment commonsense into neural networks,and proposes a Sentiment Commonsense Weight Matrix which generated by Sentiment Center to obtain sentence representation.The existing attention mechanism in sentiment classification tasks is supervised,using a large amount of labeled data to train model and learn weights implicitly.So it has the pain points of expensive training cost and low accuracy.However,the Sentiment Commonsense Weight Matrix in this paper directly utilizes commonsense for targeted learning,improving the accuracy of the attention mechanism's discrimination of key words.On this basis,this paper further proposes a Verification Mechanism,using known commonsense in the sentence to verify whether the weight matrix accurately assigns higher weight to keywords.The related work has been presented at the 2018 IJCNN(CCF-C)conference.(2)Second,this paper adopts Multitask Learning framework to integrate sentiment commonsense into neural networks,so devises an Auxiliary Sentiment Tagging Task.As the neural network embedding the semantic information of the context,it may weaken or even lose the original sentiment information of each word,which plays a decisive role in sentiment classification task.Therefore,this paper proposes an auxiliary task to control the sentiment of the hidden states: when the neural network outputs the hidden states,this multitask feeds the hidden state of each token into a tagging function and trains the tagging classifier to output the correct pre-annotated sentiment tag of this token.Besides,considering the most direct way to introduce commonsense into models as additional knowledge,this paper further incorporates the additional word level and hidden level knowledge to strengthen the effect of sentiment commonsense.The related work has been presented at the 2019 CIKM(CCF-B)conference.(3)Then,based on the above two models,this paper completes the expansion of sentiment commonsense:1)recognize new keywords through sentiment commonsense weight matrix;2)identify new sentiment words through sentiment tagging task.Finally,a closed loop from "known sentiment commonsense assisting neural networks" to "neural networks mining unknown sentiment commonsense" is formed.This paper evaluates extensive experiments on the real-world datasets to: verify the necessity and sufficiency of integrating sentiment commonsense into neural networks to improve model performance;validate the effectiveness of the neural network models in this paper with the obvious performance improvement on experimental datasets;demonstrate the feasibility of the expansion of sentiment commonsense.