The Research On Trending Topic Detection From Microblog Based On Labeled-LDA

In recent years, the convenience and openness of microblogging service make it become the main force of information dissemination quickly. In order to grasp the Internet public opinion trendings and rapidly response to emergencies, it has been research hotpots and difficulties to detect trending topics from the big scale of microblog information streams rapidly and accurately. This paper has completed several researches aim at the trending topic detection method based on topic model.Firstly, this paper analyzes and summarizes the advantages and disadvantages of existing trending topic detection method based on topic model. The research, represented by LDA topic model, includes offline pattern and online pattern. On one hand, in both patterns, LDA topic model can recognize potential topics in corpus, but topic number must be preset and the most appropriate value can make topic model distinguish topics best, so the topic number setting automatic is an important issue in trending topic detection based on topic model. On the other hand, in online pattern, prior probability of document-topic must be reinitialized in each model updating due to model each microblog as a multinomial mixture of all topics and the most appropriate value can make topic model has fast convergence speed in sampling, so update strategy is another key problem in trending topic detection based on topic model.Secondly, an offline trending topic detection method based on Labeled-LDA topic model is proposed to solve the issue that topic number must be manually setting. Bursty keywords and bursty time slices are extracted by bursty calculation method, and then the related documents are found using related tweets of keywords during bursty time slices. The labels of related documents are set according time slices, and number of labels is the topic number. After topic combining, a series of topics which consist of several words are shown. The experimental results show that proposed method in term of perplexity is always lower than LDA topic model, and make a better distinguish among topics, and in term of accuracy, recall rate and F-measure score, which is better than LDA topic model.Finally, an online trending topic detection method based on Labeled-LDA topic model is proposed to solve the issue that prior probability of document-topic must be reinitialized in each model updating. Based on the offline trending topic detection method, improved topic number estimation algorithm is proposed with considering the situation that more than one topic will appear at the same time slice. Then novel topic model update strategy is proposed that prior probability of document-topic at current time slice is initialized with posteriori probability of last time slice. The experimental results show that proposed method in term of perplexity and F-measure score, keep good performance as offline Labeled-LDA topic model and in term of running time is decreasing 34.64%.In conclusion, this paper mainly focuses on topic number estimating automatic and topic model fast and effect update strategy in trending topic detection from microblog, which not only improve the ability of distinguish topics, the accuracy and recall rate of trending topic detection method based on Labeled-LDA topic model, but also qualified the mission of public opinions monitoring.