Research On Weibo-oriented New Word Discovery And Sentiment Dictionary Construction Methods

With the rapid development of science and technology,more and more people use the Weibo platform to generate a large number of microblogs,and most microblogs contain the emotional tendency of the author.The sentiment analysis method based on sentiment dictionary is an important method to judge the emotional polarity of microblogs.However,because of the characteristics of micro-blog itself,random,colloquial and informal,it has produced many new network words,which reduced the role and significance of the existing basic emotion dictionary on microblogging tendency analysis;A large number of microblogs lead to manually creating an emotional dictionary is time consuming and laborious,and results are not effectively,So how to identify new words and automatically construct an emotional dictionary has become an urgent problem in the current microblog sentiment analysis.In response to the above problems,this paper has done three things:(1)Proposed a new word discovery algorithm based on mutual information and branch entropy.How to identify new words quickly and efficiently is a very important task in natural language processing.In view of the existing problems found in new words,A method for finding new words in the microblog corpus of uncut words from left to right is proposed.The candidate new word set is obtained by word-by-word expansion by computing the mutual information of the candidate words and their right adjacent words;The new word dictionary is constructed by calculating the branch entropy,deleting the first and last stop words of candidate new words and filtering the old words of the candidate new words and so on.The experimental results show that the new word discovery method proposed in this paper can effectively identify new words.(2)Proposed a method for automatically constructing emotional dictionary based on Word2Vec and sentence internal relationship.Automatically constructing emotional dictionary is a basic and important task in sentiment analysis.Aiming at the problems existing in constructing emotional lexicon,a method of constructing emotional lexicon automatically is proposed.Firstly,the seed word is obtained by calculating product of the word frequency and inverse document frequency.Secondly,by Word2Vec tool,using the Wikipedia as training corpus to get the word vector of words,and calculating the Similarity between the seed word and the candidate emotional word;Then using the experimental corpus used in this article as the training data,the word vector of the word is obtained again,and calculating the Similarity between the seed word and the candidate emotional word.And determining the emotional polarity of the candidate emotional words by the TwoSim method.Thirdly,obtaining the emotional polarity of the candidate emotional words by separately analyzing the micro-blog containing the conjunction.Finally,the candidate word set determining the polarity of the word is merged with the basic emotion dictionary to complete the construction of the emotional dictionary.The experimental results show that the method of constructing the emotional dictionary automatically can effectively identify the emotional words.(3)Proposed a micro-blog sentiment analysis method based on sentiment dictionary.In order to further verify the effect of the new word dictionary constructed by the new word discovery method and the emotional dictionary built by automatically constructing an emotional dictionary method,the new word dictionary and sentiment dictionary are used in the micro-blog sentiment analysis.Micro-blog usually consists of multiple sentences.According to whether the sentence contains emotional words,the emotion analysis of different methods is carried out.Finally,the emotional extreme values of each sentence are added to obtain the emotional polarity of micro-blog.