| In recent years, against the background of continuous development of distributed computing theory and big data technology and thanks to technology improvement and the support of national policy, there is increasingly wider application of distributed cluster system. The cluster monitoring system is supposed to be used conveniently for monitoring the hardware and service of the environment and nodes, dealing with abnormal situations in a timely manner. The Hadoop cluster monitoring system that is used widely at the moment alarms by sending emails and SMS to operation and maintenance staff, who have to be on the scene or connect to targeted VPN (Virtual Private Network) to check and solve the issue. This to some extent has slowed down operation and maintenance staff’s response speed and thus lowering efficiency. In order to solve the issue, Hadoop cluster monitoring system that is based on common mobile smart devices should be developed.Based on real project as well as demand of operation and maintenance staff of big data analysis and processing, this article provides a Hadoop cluster monitoring system for mobile terminals that is based on Hybrid-App. This resolution can be used on all mobile smart terminals to monitor Hadoop clusters. The detailed function of this system includes: monitoring nodes status, monitoring Hadoop service, pushing alarm notifications, configuration management, etc. The core monitoring system adopts C/S structure and it’s server terminal has integrated Ganglia and Nagio to collect indicators and alarm the engine. The system provides interface to check Hadoop cluster status and execute operation and maintenance service via RESTful API. The reverse proxy module that is built in Jersey offers Web-App of Ganglia or Nagios for mobile terminals and also it provides mobile terminal interface and Agent terminal interface, which are for the communication between application on mobile terminals and system agent. The system Agent terminal, which is deployed on monitored nodes, reports status info of itself and nodes to system server via heartbeat connection; meanwhile it will read action queue that is buffered on the server to execute actions in the queue orderly on location. The cluster monitoring interactive interface on mobile terminals is uploaded in the form of Web-App to mobile terminals. This mode can simplify cross-platform development process and to a large extent it can help solve fragmentization issue on Android platform. Taking into consideration of big data safety, this article has designed and put into practice a lightweight security channel with reference to SSL (Secure Socket Layer) protocol.The author has participated in the whole process from demand analysis to design and development of the system. The main responsibility was decoding of system server’s security module (including authorized module, data encryption and decryption module and key management module), server’s mobile terminal interface module, action queue module and action executor module of system agent terminal, security channel protocol package and design and realization of app on Android platform. Meanwhile the author was also involved in development and testing of other modules in the system and deploying the original version of this system in actual production environment.The Hadoop cluster monitoring system based on mobile smart device that is elaborated in this article is now being tested internally. Operation and maintenance staff can receive alarm notifications from Nagios on mobile platforms; also they can check out the cluster monitoring interactive interface on any handy smart devices, thus their respond speed and working efficiency can be significantly improved. |
PDF Full Text Request