Research And Application Of Security Event Mining Technology Based On System Log Analysis

With the advent of the era of big data,the scale of the network continues to expand,and related methods and technologies for maintaining the security of cyberspace are becoming more and more important.Among them,all kinds of computing systems running in cyberspace generally build and maintain system operation logs in text,and the information recorded contains a large amount of local security event information.Because security information has the following characteristics:comprehensiveness,full time domain,irreplaceability,etc.,mining security events and related technology research based on system logs has become cyberspace security.Therefore,here to improve the speed and performance of security events based on system logs Objective,carry out related research,design and implement a security event analysis system based on system logs.The main research work and results include:1?The requirements and conditions of telecom network application scenarios are analyzed in detail,and the characteristics of network security events,text semantic feature extraction methods,text classification methods,and parallel computing architecture are studied in detail.2?In order to improve the accuracy of log data classification,through research and analysis,combined with BERT and LSTM algorithms,the analysis and classification of security events in the system log is realized.The main feature is that the traditional language model can not solve the problem of word ambiguity.It is proposed to use the BERT pre-trained language model to extract the semantic feature information of a single log,and combine LSTM recurrent neural network as the training network.In order to improve the accuracy of log data classification,through research and analysis,combined with BERT and LSTM algorithms,the analysis and classification of security events in the system log is realized.The main feature is that the traditional language model can not solve the problem of word ambiguity.It is proposed to use the BERT pre-trained language model to extract the semantic feature information of a single log,and combine LSTM recurrent neural network as the training network.Experimental results on a provincial telecommunications company's real data set and IDS2018 data set show that it is superior to other text classification models.3?Based on the concept and technology of parallel computing,a two-level parallel computing architecture is designed,which applies Spark computing technology,Kafka distributed messaging system and GPGPU parallel processing.This architecture can effectively improve the calculation efficiency of log data analysis and mining.4?In order to verify and apply the above research results,based on the telecom network platform,a design scheme of "log-based security event analysis system" is given.The system includes log preprocessing module,Kafka parallel computing management module,GPGPU parallel computing management module,log calculation analysis module and other functional modules,which can combine telecom network platform management business requirements to achieve network security event analysis and classification.Main research contributions:BERT-LSTM log data classification algorithm analyzes and classifies network security events,and builds a two-level parallel computing architecture based on Spark computing technology,Kafka distributed messaging system and GPGPU,and implements a log-based security event analysis system.The application system is implemented and tested.The results show that the system can effectively achieve the expected function.