Design And Implementation Of Social Media Data Analysis System

With the rapid development of social media, people are using social media to share their life stories, to express their points on things, etc. As Sina Weibo and other social media generate a lot of first-hand consumer information, more and more data becomes fingertips. Currently, many companies just put social media as a marketing channel, and didn’t take full advantage of data generated by social media. Sina Weibo is regarded as the data source. Firstly, based on the distributed data collection methods, the platform utilizes the APIs of Sina Weibo to access to the data about users. Then, these huge amounts of data be stored and analyzed in Hadoop. This system realizes the combination of Hadoop and Solr, and gave full play to the advantages of both. The index of Solr search engine using Hadoop pre-segmentation technology and distributed structure has been optimized and efficient. Finally, after a massive social media data analysis, the system completes the visualization of analysis result.According to the data generated by Sina Weibo, the system was designed and implemented. The system is mainly responsible for the acquisition and analysis of massive social media data. Constructing Hadoop platform to realize the distributed computing system of the acquisition of social media data and building website to analyze massive data are regarded as main task. The function of data analysis includes social relation network analysis, group analysis, emotional analysis, customer city map, topic trend analysis, etc. This paper expounds the background and the development trend of the project, introduces the social and commercial value of the system. And then, the relevant technologies used in this system are introduced in detail. Immediately following, this paper introduces the business and function analysis of the system, further expounds functional and no n-functional requirements. On the basis of discussion above, the architecture of the system was designed, each function module was divided. The author participated the software development life cycle including requirement analysis, outline design, detailed design, coding and testing. The author mainly realizes social relation network analysis module, group analysis module, emotional analysis module, customer city map module. The system had finished the test inside the company. Those modules designed and implemented by author have been running normally...