Big Data Analysis Of Autonomous Container Cloud Platform Logs Research And Implementation

Service information system is constantly transforming into a networked information model,and domestic hardware equipment is constantly b eing updated.Self-control is also a basic requirement in new information age.With the development of the information age and the independent control requirements of the new era More and more services and applications will also be deployed on autonomously controllable cloud platforms.During the operation of the services and applications deployed on the autonomously controllable container cloud platform,we need to obtain log data for a series of monitoring,statistics,analysis and forecasting work.However,traditional log collection and analysis tools have some technical drawbacks for autonomously controllable container clouds.The purpose of this paper is to build a big data log analysis system for the autonomously controllable container cloud platform.This paper mainly uses the data log processing technology with Spark and ELK Stack open source fram ework as the core,and realizes big data analysis for the system log of the whole autonomous controllable cloud platform.The log analysis system of the entire autonomous container eloud platform is divided into four parts.In order to adapt to the characteristics of the self-controllable container cloud,this paper designs a log collection algorithm for adaptive system load,which reduces the impact of log collection on the performance of the container cloud platform.In order to perform log analysis tasks efficiently,this paper conducts detailed research and product requirements planning for the log data analysis stage,analyzes the factors affecting Spark performance in detail,and optimizes the performance of Spark clusters based on the characteristics of autonomously controllable container cloud.Based on the above research results,this p aper discusses the implementation process of big data analysis of the autonomous container cloud platform log.The big data log analysis system of the autonomous container cloud has been tested and used in engineering projects.