Research And Implementation Of Real-time Security Log Analysis System For Cloud Computing Platform

Cloud computing is an important basic information facility of the country,supporting complex systems such as e-government and financial securities.Cloud computing platform security and health management is the premise and basis for its wide application.Along with the cloud computing service process,a large number of logs such as system operation,security protection,and device status are generated.Multi-source heterogeneous massive cloud computing log analysis is an important means of security and health management.The cloud platform security log format has many types,low semantic level,and non-uniform information.The traditional log analysis method cannot meet the requirements of cloud computing platform security analysis in terms of accuracy,real-time and efficiency.This paper focuses on the accurate analysis of multi-source heterogeneous security logs and the active defense based on the association analysis of massive security event rules.The cloud platform security log real-time analysis system is designed and developed.The log analysis method based on cluster classification only extracts the key information of the log,and cannot meet the requirements of the cloud computing security platform in terms of log conversion accuracy.Aiming at the characteristics of cloud platform security log format,low semantic level and inconsistent information,this paper proposes a multi-source heterogeneous security log parsing method based on streaming architecture,which realizes the standardization of multi-source heterogeneous security log with low semantic level.Structured processing to support log correlation analysis.The machine-based massive security event batch processing method cannot meet the requirements of the cloud computing platform in terms of real-time and accuracy due to lack of consideration of time correlation.The single rule association is transformed into multiple hybrid rule associations,and the association analysis rules are formulated according to the security analysis scenario.A multi-layer rule association analysis method based on time dimension is proposed to realize automatic association analysis of security events in multiple scenarios.The actual test shows that the average delay of security event analysis is less than 1 second,which meets the real-time requirements of cloud computing platform security analysis;the multi-layer rule correlation analysis method reduces the false positive rate by 38% and improves the accuracy of security event analysis.Based on the researched techniques and methods,a real-time security log analysis system based on streaming architecture is designed and implemented,which consists of a rule association analysis subsystem,a log management and analysis subsystem.The rule association analysis subsystem adopts a multi-layer rule association analysis method based on time dimension to realize visual analysis and automatic association analysis functions,which meets the requirements of log analysis accuracy,real-time and high efficiency.The log management and analysis subsystem adopts a multi-source heterogeneous security log parsing method based on streaming architecture to realize real-time log collection,log index query and visual analysis.The system has been applied to the Dragon Stack cloud platform to verify the availability of the technologies and methods studied and to support the security analysis of the cloud computing platform.