Design And Implementation Of Microblog Interaction Platform Based On Storm

The past decade has seen a revolution in data processing. MapReduce, HDFS, andrelated technologies have made it possible to store and process data at scales previouslyunthinkable.Unfortunately, these data processing technologies are not designed for real-timeprocessing of large data. With the rapid demand changes in the business operation,unceasing innovation in the business model, personalized and real-time demands in userexperience, requirements for realtime massive date processing is becoming more and moreprominent. The lack of a "Hadoop of realtime" has become the biggest hole in the dataprocessing ecosystem. Twitter Storm fills that hole. How to use the real-time, distributed,streaming processing framework Storm to improve the performance and efficiency of thesystem is what this paper focuses on.The paper is based on the research project―yeezhao micro-blog interactive platform‖.Some analysises and researches are performed to deal with the problems of time-cosumingdata processing and low system resource utilization. The paper uses the softwareengineering methods, applies agile iterative development approach and object-orientedanalysis and design related technologies, to describe the aspects varying from requirementsanalysis, preliminary design, detailed design, concreted implementation to systemdeployment, and introduces a variety of techniques and methods used in the procedure ofsystem design and implementation. Finally, a real-time data analysis and processing systemis designed and implemented. Some experiments are performed to test the system’sperformance and harizontal scaling effects against the previous one. According to the experimental results and application effect, the system based on Storm shows up its higherperformance and instantaneity, as well as its stronger scalability and availability on the dataanalysis and processing other than the implemention based on traditional solution.