The Research And Implementation Of Distributed Monitoring Platform Based On Hadoop

As an emerging hotspot in technology, cloud computing has developed at an unprecedented speed, extending to all walks of life, as well as changing the technical architecture and operation model of enterprise. Nowadays, a lot of enterprises, research institutes, universities and governments have taken the initiative to explore this area, applying it to the production. In order to ensure the stable operation of cloud servers and the various cloud computing services, the major cloud computing vendors and third parties have launched their own cloud monitoring services. As a basic service, Cloud Monitoring focuses on monitoring the resource use of cloud servers and cloud computing services, running state, availability, etc. What’s more, it also offers some advanced warning services or data analysis services based on the monitoring data.Cloud monitoring aims to master the real-time performance and the running state of cloud resources, responding to the abnormal problems quickly, to provide users with a safe and stable service. Therefore, on the basis of the theories in cloud computing and cloud monitoring key technologies, this thesis researches into and designs a perfect monitoring platform, which is responsible for resources guarantee, application state, site running state, availability and log analysis. In this way, users can pay more attention to their own business services.In this thesis, cloud monitoring is divided into process monitoring, site monitoring and log monitoring on three levels. In terms of function, it is divided into distributed data acquisition, monitoring mission control, monitoring data analysis, monitor abnormal alarm, task scheduling center and monitoring display. Based on the basic monitoring, this thesis is going to combine cloud monitoring with data analysis technology, analyze the collected monitoring data, mine monitoring information, as well as provide exception analysis, warning service or other data analysis services. From the perspective of monitoring, this thesis will provide warning, analysis and prediction of abnormal service, in order to achieve a more stable and meaningful cloud monitoring for the users.