Research Into Monitoring Information Collection And Storage Techniques Of Large-scale Cluster Monitoring System

Cluster Monitoring System (CMS) is a kind of very important infrastructure of large-scale cluster, which are monitoring the health states and performance indexes of the cluster, and providing the functions for diagnosing the run-time failure and exception. The elementary tasks of CMS include: collecting the needed information from the monitored nodes, analyzing these information, and finally making decision according to the analyzed results. On the techniques of collecting and storing of monitoring information, the key techniques of CMS, this thesis gave an intensively study. The collecting component of CMS consists of two units: monitoring information probing unit (MIPU), which collecting needed information from the monitored nodes, and monitoring information influx unit (MIIU), which putting all the information together so as to analyze and process synthetically. Moreover, the monitoring information storage unit (MISU) focuses on storing all the monitored information in order to analyze the historical data and to the other application systems be accessed by. Therefore the research contents of this thesis are further divided into three parts :MIPU,MIIU,MISU. On the whole, a good CMS should decrease its resource consumption, reduce its impact on the cluster, and improve its reliability, stability and extensibility. On account of different locations and different work, the three units, MIPU, MIIU and MISU, have some different evaluation index and scale. In this thesis, three kinds of evaluation index and scale are given, and various analysis and evaluation are made in accordance with each unit. As for MIPU, this thesis focuses mainly on how to collect the kernel information from the node machine, and a collecting method based kernel module is proposed and deeply studied. With regard to MIIU, the communication pattern, communication model, and primal communication protocol is studied in the most, and through evaluating them synthetically, a best MIIU implementation scheme is proposed. About MISU, this thesis places emphasis on synthetically evaluating the storage location, structure and organization of the monitoring information. Accordingly, a best MISU implementation scheme is proposed. The project background of this thesis is SunEye6800, which applied to lenovo DeepComp6800 cluster system. In this thesis, the related research and implementation work on SunEye6800 is introduced in detail, in which the concrete implementations of the three units MIPU, MIIU, MISU are discussed. Finally, some conclusions are drawn according to the results of experimentation and the future works are presented.