The Research And Implementation Of Public Opinion And Analysis Platform Based On Microblog Data Mining

Recently,with the rapid development of Internet,it makes people to face the immensity data at loose ends.At the same time,network security issues become increasingly prominent,the frequently appeared emergencies result in tremendous social loss,therefore more and more attentions have been paid by people.Microblog is a new kind of Internet media platform that appeared in several years ago,Microblog is becoming increasingly important in our life.But on the other hand,there are some disharmony and uncivilized behavior,and even some expressions of anti-government and disrupting society.For what have been talked above,Microblog public opinion warning and detecting technology came into being.To correctly guide public opinion,clean network environment,the government or relevant government departments need to provide a new proficient management means,the public opinion detecting and analysis platform is what they need.This system's main goal is to provide a service that public opinion detecting and analysis for relevant government departments guide public opinion by crawl the Microblog web to obtain web Information,save the data and then display the results after data analysis.The system is mainly consisted of the following three parts:The first part is information collection.Through the studies in the Open Source Framework Nutch,Crawler and Web Collector,this system will design a multithreaded web crawler for Microblog to crawl the microblog web information and save it combined with the best of them.The second part is data analysis.This paper will contribute a method,which aims at identify hot topics in Microblog based on k-means.In this method,after the pre-disposition of the Microblog data,by dividing time-window,by extracting topic words according to the two factors of increasing rate of word frequency and relative word frequency from Microblog data in every time-window,then sieving for a suitable cluster of topic words so as to describe the hot topic,we hope to realize the detection of hot topic in Microblog.The third part is public opinion visual representation.The system will display the hot topic and trend of public opinion by the results of second part.