Adaptive Kubernetes Network Performance Monitoring System Based On Dynamic Service Topology

With the rapid development of cloud native technology system represented by Kubernetes(hereinafter referred to as K8s),especially cloud native applications are closely combined with K8 s,applications are becoming more and more distributed,micro-service-oriented,highly scalable,multi-instantiated,and portable.Due to the characteristics of application,business services become loosely coupled and distributed discretely,locating and troubleshooting the fault points in cloud native application environments become difficult.Monitoring on K8 s faces a huge challenge.In K8 s environment,there are two dimensions which are necessary to monitor: system resources and network.Nowadays,the community has produced many tools for monitoring system resources(such as CPU,memory and storage),which are becoming more and more mature.However,at the network level,they are not well-suited for monitoring container network due to the complex characteristics of K8 s virtual network.The traditional network monitoring tools become ineffective,thus it's necessary to created a new method and tools to adapt for K8 s network monitoring.There are some problems related to the collection and analysis of network data packets on K8 s network monitoring as following.Firstly,due to the diversity of K8 s CNI network plug-ins,the monitoring scheme needs to be adapted and compatible for different network plug-ins with different network protocols and working mechanisms.Secondly,due to the diversification of K8 s CRI container runtime and the drift of Pod,the acquisition(adapting to multiple CRI implementations)and dynamic identification of Pod's containers are problems needs to be solved.Thirdly,at present,there are some monitoring schemes for k8 s container network,which still focus on the collection and sorting of IP layer packets.IP especially Pod's IP is extremely unstable and dynamically changeable in k8 s environment,while the traditional server's IP rarely changes.Therefore,the timeliness of analyzing packet metrics based on the information of network layer and transport layer is reduced,and the traffic analysis will be very complex because of the difficulty of reading and identifing IP address.If the network data packets can be sorted out for traffic analysis from the perspective of business or service,the labour power and time spent by the operation and maintenance personnel in troubleshooting will be greatly reduced.Aiming at the problems mentioned above,this paper designs and implements a K8 s network performance monitoring system based on dynamic service topology mapping in cloud native environment.The system realizes adaptive compatibility with a variety of K8 s network plugins,adaptive perception of Pod,dynamic monitoring of Pod changes,self adaptation to different container runtime,and establishes Pod to Pod network data collection based on dynamic service topology mapping method proposed.The collected data packets are made a correlation analysis,which will form metrics and be reported.The system is divided into two modules: collector and controller.The collector is used to collect and report the data metrics of K8 s network,and the controller is responsible for the storage,analysis and display of metric data.This article realizes the full coverage of end-to-end(Pod to Pod)traffic collection in the large K8 s container network environment.Dynamic service topology mapping makes it easy to establish a complete network structure topology.By summarizing and analyzing data packet correlation metrics data,it is very easy to draw network transmission paths and communication paths between services,which helps to solve the problem of monitoring and identifying network data from the business dimension under the large K8 s container network.it can help operations engineers,developers,product managers and other personnels to locate the network fault points,and troubleshoot the network performance problems for cloud native application in K8 s environment.