Unsupervised Keyword Extraction For Microblog Posts Based On Hashtag

Nowadays,a huge amount of text is being generated for social networking purposes on the Web.Keyword extraction from such text benefits many applications such as advertising,search,and content filtering.Unlike traditional web pages,a microblog post,usually has some special social feature like a hashtag that is topical in nature and generated by users.Extracting keywords related to these hashtags can reflect the intents of users and thus helps us better understanding on post content.In this thesis,we propose a novel unsupervised keyword extraction approach for microblog posts by treating hashtags as topical indicators.Our approach consists of two hashtags enhanced algorithms,and the Specific work is as follows:(1)One is the random walk based algorithm.Firstly It builds a word-post weighted graph by taking into account posts themselves.Then,a hashtag biased random walk is applied on this graph,which guides the algorithm to extract keywords according to hashtag topics.Last,the final ranking of a word is determined by the stationary probability after a number of iterations.(2)The other is the topic model algorithm which infers topic distributions biased to hashtags on a collection of microblog posts.The words are ranked by their average topic probabilities.The topic model algorithm can not only find the topics of a collection,but also extract hashtag-related keywords.(3)We evaluate our proposed method on a collection of real Chinese microblog posts.Experiments show that our approach is more effective than traditional approaches without considering hashtags in terms of precision.The result combination of two algorithms performs further better than individual algorithms.(4)In this thesis,The visualization technique is applied to analyze keyword extraction and public microblog hot topic.The word cloud visualization for microblog was designed and used to research the information of users in period of time and characterize the differences from the perspectctive of topic and time.The keywords,hashtags in microblog hot topic are displayed through word cloud visualization,which is beneficial to a more intuitive visual effects for users.By this way,users can obtain the important information under a hot topic more briefly and easily.