A Real-time Analysis System Of Potential Demand For Weibo User In Storm

With the development and popularization of the Internet,microblogging,as an open platform for exchanging and sharing information,can produce hundreds of millions-level data.It is a precious wealth for enterprise by mining these massive data and analyzing the potential purchasing power of users.However,the current analysis method has some drawbacks:lag and untargeted.In this thesis,real-time analysis platform,which called storm,will be used to improve existing research methods,while addressing lag and other issues.Firstly,the scheduling strategy of Storm will be optimized and using the optimized Storm to set up a real-time analysis system to excavate the potential purchasing power of microblogging users.Specific tasks include:list questions of Storm caused by the average scheduling assignments,verified through the experiment,and then realize scheduling model to solve the time delay and poor efficiency of the message inside the node local characteristics.Followed by the design and implementation of real-time analysis of the system:using spilder and Sina API data to solve source issues;build Kafka cluster,coordinating with the data source module and data processing module in real-time;using Spout and Bolt of data to design and implement the data module and data analysis module;data storage module using for data persistence,using SpringMVC and ECharts complete data display module,to analyze the results show.Experimental results show that the improved scheduling strategy performance is significantly better than the default scheduling policy,especially in terms of CPU-intensive tasks scheduling,performance increased by more than 50%.Real-time analysis system can analysis of the potential buying behavior of users in real time,fully satisfy the needs of enterprises.