Document-level Sentiment Classification Based On Dynamic Word Embeddings And Hierarchical Neural Networks

Due to the recent rapid development of computer technology and the comprehensive popularization of Internet facilities,massive amounts of text data have been produced by Internet users on online platforms.These text data play a key role in many applications,such as e-commence website,public opinion analysis and interest mining,etc.In order to explore the fruitful sentiment information from text data,researchers have applied various kinds of methods,document level sentiment classification is one of them.Sentiment classification,known as predict the sentiment polarity of text data,has been mature applications in many scenarios.However,current research still has two major challenges: First,the expression of emotions in the network platform is flexible and changeable,and the semantics of words often vary with the context.Current word representation methods statically map words into one single vector,which failed to capture contextual meanings for polysemous words.Second,the existing sentiment classification methods often treat the document as basic input unit,which cannot make full use of the hierarchical structure of documents.Aiming at these problems,this thesis studies document level sentiment classification methods based on dynamic word vectors and hierarchical neural networks.The innovations of this thesis are summarized as follows:(1)This thesis applies ELMo(Embeddings from Language Models)to generate dynamic word vectors,which can produce the actual semantics according to the context of the word.As a result,it solves the problem of polysemous words in the traditional word representation methods.Besides,this thesis proposes a hierarchical model named HieNN-DWE that combines BiGRU(bidirectional gated recurrent neural network)and CNN(convolutional neural network).HieNN-DWE can not only make full use of the hierarchical structure of the document,but also can mining the global feature dependence in the document and the semantic relationships between sentences through BiGRU and CNN.(2)Aiming at the limits of ELMo,this thesis proposes a novel dynamic word representation method based on sentiment dictionary,dubbed as SL-ELMo(Sentiment Lexicon based ELMo).SL-ELMo explores the sentiment information from words by applying the sentiment lexicon and masked language model at input layer.Besides,SL-ELMo can capture the contextual semantics of words by applying bidirectional gated convolutional neural network.Such improvements make SL-ELMo more suitable for sentiment classification task.Experimental results show that SL-ELMo can further improve the ability of capturing semantic representation and contextual information from words.(3)This thesis designs a Chinese text sentiment classification system,which is easy to use and highly available.Besides,this thesis conducts experiments on Chinese review dataset,results proved the effectiveness of this model in the Chinese environment.Hence,the model can be applied to real production environments,which has theoretical and practical significance.