On The Discovery Of New Words On Weibo And Judgment Method About New Words' Sentimental Polarity

The development of the Internet has constantly changed the way of communication and expression of human beings,which leads the emergence of numerous new words.As the most popular social networking media in the context of big data,Weibo has becomes an important platform for the emergence and rapid spread of new words.The ineffective identification of new words has influenced the accuracy of Natural Language Processing on segmentation and sentiment analysis of Chinese words to a great extent.Therefore,the discovery of new words in texts of Weibo and the judgment of sentimental polarity are of great significance and value.The sentimental polarity judgment of words is the basis of emotion analysis in the research of sentiment analysis.New words are widely used due to their short character and concise way of expression.Because of the lack of understanding of new words,it is difficult to judge the sentimental polarity of new words,thus affecting the effect of text sentimental analysis.The first step to complete the above research is segmentation,which directly affects the correct rate of sentiment analysis.The new word is the main factor affecting the accuracy of the segmentation.So,the study of this thesis is based on the discovery of new words on Weibo and judgment method about their sentimental polarity,the research contents are listed as follows:New word discovery based on improved new word synthesis algorithm is proposed.First,the problems of some new words discovery methods have been analyzed.The new words that are misclassified into multiple words by the word segmentation tool are combined with the statistics of multiple words point-wise mutual information,the left and right branch entropy,and the improved new word synthesis algorithm in this paper.The new words are merged with adjacent words to obtain the candidate new words.Then,the new word sets are collected by low-frequency word filtering,stop word filtering,word formation rule filtering,common sentiment lexicon filtering,and other post-processing.Sentimental polarity judgment based on improved semantic orientation point-wise mutual information is proposed.At first,the words obtained by text segmentation are converted into vector form,and the similarity between new words and other words is calculated to obtain several similar words of new words.The semantic orientation pointwise mutual information is improved by the emotional polarity value of basic emotion words.Then,the possible emotional tendencies of new words can be deduced,by calculating the semantic orientation point-wise mutual information between the new words and the emotional words in the set of similar words.This thesis holds that the improved method of new word discovery studied in this thesis can extract new words effectively.The improved method of new word sentimental polarity judgment can improves the ability of the new word's emotional orientation recognition.Judging the sentimental polarity of new words can improve the effect of sentimental analysis of sentences in Weibo.