Short Text Modeling Based On Two-level Attention Networks For Sentiment Classification

Sentiment Classification of short texts has always been a very challenging problem in the field of information processing.Unlike documents,short texts have some unique characteristics which make them difficult to handle:(1)Short texts contain limited context,and the semantic expressions of short texts are often incomplete.(2)Short texts generated by users,especially search queries,posts do not always observe the syntax of a written language.This means traditional NLP techniques,such as syntactic parsing,do not always apply to short texts with good results.To improve the results of sentiment classification of short texts,traditional NLP models generally represent short texts with vector space model and optimize the classification results by extending text features.However,since short texts are generally sentence-level texts,most of them face the problem of data sparse.The performance of traditional NLP models tends to be average.Therefore,this paper applies the deep learning to analyze the sentiment of short texts.The main work of this paper is as follows:(1)Through the overall analysis of the current methods of sentiment analysis of short text,the models based on deep learning are the most advanced methods.Based on the existing models based on deep learning,we find that: 1)Some short texts contain one or more words that are inconsistent or highly different from the actual expression of the short text,which we call "noisy words"."Noisy words" interfere with the judgment of the deep models,and they lead to an increase error rate of deep models.These "noisy words" include: interference of unregistered words,interference of polysemy,and interference of description objects.2)When there is a long-distance special language structure in some short texts,the deep models have low ability to capture the language structure features.It also leads to an increase error rate of deep models.In the work of this paper,we have in-depth exploration of the causes of the two kinds of problem.(2)To address this,we propose a novel model based on two-level attention networks to identify the sentiment of short text.Our model first adopts attention mechanism to capture both local features and long-distance dependent features simultaneously,so that it is more robust against irrelevant information.Then the attention-based features are non-linearly combined with a bidirectional recurrent attention network,which enhances the expressive power of our model and automatically captures more relevant feature combinations.We evaluate the performance of our model on MR,SST-1 and SST-2 datasets.The experimental results show that our model can outperform the previous methods.