| With the rapidly development of Information technology,Internet social application has became a extremely important information exchange platform of people who in the new era.The way to discover hot spot topic from massive data,No matter in the commercial analysis and promotion,or public opinion analysis,user interest location and content recommendation,it takes great meaning on both reality and application.Therefore this paper designs and implements a real-time online hot spot topic generation system.This system analysis the topic discover process and decompose the task,implements the automatic process which includes source corpus scanning,hot spot topic generation and result visualization.At the same time this paper also designs and implements data scanning sub-system,hot spot generation sub-system and result visualization sub-system,the online and offline corpus based on the third party storage component,and corpus relational database,result relational database.In the topic generation sub-system,this paper proposes a top-down text clustering topic generation algorithm based on text entropy and Unigram text modeling.System supports the application on different social platform in programming way,it parse different data to time-document-topic structure,and it also supports statistics and query with different dimensions in the result visualization sub-system.Based on the content above,this paper applies this system to Hupu,a domestic social application,for satisfying Hupu's requirement of lacking content mining hot spot generation function and users' hot interests location.With the analysis of result visualization and sub-systems' task accessing performance on Hupu data,it proves that the real-time online hot spot topic generation system can efficiently and stably access the process of hot spot topic generation in the limited compute resource and acceptable compute time environment,and it eventually gets a good hot spot topic generation result. |
PDF Full Text Request