Research And Implementation Of Cloud Computing Platform Monitoring System Based On Twitter Storm

Cloud computing is the extension and development of grid computing, parallel computing and distributed computing. It can solve the difficulties of processing big data effectively in the information age. Hadoop is the current mainstream cloud computing platform. By ingregating tens of thousands of cheap computers and servers, it can provide much more computing power and storage space. At present, cloud computing technology has entered the stage of implementation, many companies begin to build their own data centers, to deploy and operate cloud computing platform. However, thousands of servers bring great challenges to the management. A stable, reliable and real-time monitoring system is becoming the foundation.This paper has built the monitoring system based on Twitter Storm relying on the National Marine Cloud Computing Project. Users can make subscriptions to the monitoring cluster in web page according their own demand by graph and list. This paper has also put forward the corresponding optimization strategies about slow speed and data loss issues.According to the marine project, data resources, computing resources and virtual resources are needed. This paper firstly designs the system’s architecture and lists the system frame, and then explains the content from data collecting module, data processing module and platform displaying module one by one. Data collectting module is the basic source of the system data stream. It makes use of the technology of Ganglia and collects the real-time data according to the subscriptions, then delevers the data by XML. Data processing module is the core part of the system. This paper designs the topology model from several modes. The module can compute accurately and return the result in real time. In addition, the module provides ad data persistence function, and then users can analysis the history of cluster by it. Platform displaying module is the connection window between users and the system, users can directly monitor the cluster’s situation on the interface and manage the hosts in finer granularity by setting subscriptions.Finally, this paper introduces basic data stream processing model based on the data stream concept model, then proposes the data processing strategy based on Storm. It effectively guarantees the real-time data to be processed completely and data stream to be grouped by order. This paper also optimizes the traditional sliding model and designs the aggregate query based on composite sliding window model. System evaluation introduces the optimization much more, and displays the results graphically. It indicates the optimization algorithm extensibly and practically.