Design And Implementation Of Cluster Monitoring System For RPC Service Monitoring

The nodes in cluster system typically communicate with each other through the RPC(Remote Procedure Call,RPC)service.However,the probability of failure of the RPC service is greatly increased due to the cross-host communication.Thus,the monitoring of RPC service is an important part of cluster monitoring.The monitoring data is huge and its update frequency is fast during RPC service monitoring.The traditional open source cluster monitoring system suffers from low RPC monitoring dimension,inefficient means to collect data,and it also is not able to process large-scale monitoring data in real time.It is of great significance to study and design a cluster monitoring system for RPC service.Based on the operational requirements of distributed clustering system and the development framework of RPC Service in Company A,this paper studies the monitoring methods of RPC service.Considering the shortcomings of traditional cluster monitoring system,this paper designs the cluster monitoring system for RPC service based on the idea of hierarchical design,which mainly includes:1.Aiming at the lack of RPC service monitoring dimension in the traditional open source cluster monitoring system,this paper designs a data collection agent based on the lock-free shared memory queue.Benefiting from the lock-free shared memory queue,the data collection agent ensures efficient communication between the monitored process and the data collection agent process.It achieves a low-overhead collection and reporting for the RPC service monitoring that has fast update frequency and large amount of monitoring data.2.This paper realizes a high-concurrency gateway server based on reactor event-driven model and thread pooling technology to solve the problem of inefficient data collecting.The server can respond to a large number of concurrent requests for data report from different monitoring nodes,and make efficient collecting for monitoring data.3.The paper designs a monitoring data computing center based on the real-time data processing framework Apache Storm.The data computing center is able to process large-scale monitoring data in real time by Storm computing task.By testing and analyzing the system,this system achieves the low overhead collection,high performance collecting and low delay processing of RPC monitoring data.Thus,the system can effectively monitor the RPC service and cluster nodes in real time.