| With the rapid development of the social network, microblog has gradually become a platform of information communication in daily life. In a very short period of time, the microblog platform can generate vast amounts of data sets. Microblog users are difficult to distinguish hot events from these massive text messages, therefore, how to quickly and accurately excavate hot events from the massive microblog text data sets has become the focus of current research. Because the traditional topic detection methods for microblog is usually based on feature words matching,it also don’t consider potential semantic of microblog text, finally which lead to the poor quality of topic detection. According to the characteristics of microblog,from the perspective of semantic this paper makes a deep research on hot event detection for microblog ,an topic detection method based on improved H-K clustering algorithm is presented.First, according to the characteristics of microblog data sets and topic, this paper improve H-K clustering algorithm which is used in method of topic detection. In view of the massive microblogg data sets,in order to improve the processing efficiency of clustering,in the method of microblog topic detection this paper combined with MapReduce programming ideas in Hadoop, the algorithm realize parallelization. Then, in order to reduce the dimensions of the microblog text and improve the accuracy of the microblog text similarity calculation,this paper carries on the analysis to the microblog text from the semantic level, the LDA topic model is introduced, the model makes the unstructured microblog text into text-themes distribution and theme-words distribution. Meanwhile in the microblog text modeling phase,according to MapReduce parallel programming ideas, LDA topic model achieve parallelism,so as to improve processing ability of microblog data sets.Experiments show that, the clustering effect of the improved H-K clustering algorithm can be improved, the improved algorithm not only has high accuracy and low time complexity of the algorithm,which also solve the problem that efficiency of traditional clustering algorithm is not high, in the clustering of microblog text stage this algorithm has a very good application; The introduction of the cloud computing platform improve the processing capacity of microblogg data sets; The method proposed in this paper can fast and accurately excavate hot events from the massive microblog text data sets. |
