Design And Implementation Of Hot Spot Discovery And Public Opinion Analysis System Based On Big Data

When doing realistic news production,there are many bottlenecks in the production environment that constrains the production of news content.For example: short-term hot spots can not be captured,editing manpower is limited,the relevant material is difficult to collect,a lack of reasonable feedback mechanism.The media needs a tool that can find hot spots,provide material support and track hot spots.The Internet has become the next generation of information media,its development speed and influence has been far more than the traditional newspapers,radio,television and other mainstream media.The Internet has become the ideological and cultural information distribution center and the amplifier of the public opinion,which makes public opinion monitoring for enterprises,organizations,institutions very important.Hot spot and public opinion discovery and analysis system uses Hadoop computing platform to analyze large data.Hadoop computing platform is used for hot spots and public opinion analysis.Hot digging through a period of time news data finds hot spots.Public opinion analysis use the comment data and social data associated with hot spots.When doing emotional analysis,point of view and user portrait is used for public opinion analysis.All data is stored with the Hadoop storage platform.We index news data,and using the retrieval system to provide material retrieval services.Finally,web pages are presented for the media to provide hot spot discovery and clue management functions,provide public opinion analysis and alarm function for enterprises,organizations,institutions.System gets news and comment data and Sina microblogging data through the download platform,gets social and search data by internal push process.The preprocessing mainly includes geographical classification,domain classification,low quality filtering,emotional analysis,site identification and authoritative media certification.After that,a piece of data will be stored in the Hadoop cluster,and the other will be indexed.Index data can be used for public opinion analysis or for material retrieval.After the use of algorithm components for hot spots and public opinion analysis,get hot topics and public opinion related data.Using Hadoop files and MySQL database for data interaction.By web form,according to different business needs to present data.Hot spot and public opinion discovery and analysis system version 1.0 has been accepted and confirmed by the People's Daily.Of course,there is a need for a perfect system.