Research And Application Of Monitoring And Warning Technology For Large Scale Cluster Management Platform

In recent years,with the advent of the information age,the data show explosive growth trend,the ordinary large computer has been unable to bear the task of calculating the mass data,the major Internet Company have adopted large-scale Hadoop cluster to complete data storage and analysis.With the Hadoop cluster is becoming more and more larger,to ensure the stable operation of large-scale cluster has become a focus of attention.In order to monitor the running status of the clusters and the problems in the cluster,it is needed to develop a set of tools for monitoring and warning.This tool can provide a visual front interface,and support automatic deployment and expansion of monitoring and let the controller manage the cluster effectively,to ensure the normal work of the cluster.Firstly,this thesis introduces the background and significance of this research,and makes an investigation and introduction of the existing cluster monitoring and warning system,and illustrates the characteristics and application value of the monitoring and warning system.Secondly,the thesis introduces the basic knowledge of the distributed cluster,the three key problems which are considered in the design of the monitoring and warning system,and the corresponding solution and design idea.Then it describes the overall structure of the monitoring and warning system,the operation flow of the monitoring platform,and introduces the various modules of the platform.Again,focus on the system's two main components,the collection module and alarm module.First of all,we describe the collection module,start with the structure of the collection module,then we give a detailed description of the working mechanism of the collection module.Followed by the structure and work flow of the alarm module,including the monitoring indicators etc.Then we extract the performance data from the database and using them to draw figure for analyzing the alarm items of the cluster.Finally,the thesis has done a summary of the research,and put forward the problems in the system and the future needs to improve the system.