Research And Application Of Log Anomaly Detection Method Based On Gated Recurrent Unit Network

Log anomaly detection is one of the research hotspots in the field of network security fault diagnosis.Deep learning technology is a novel application of log anomaly detection.In order to improve the performance of log anomaly detection method,this paper studies log parsing and anomaly detection algorithm from two perspectives.A distributed architecture is introduced into log parsing,combining execution path characteristics and parameter characteristics,and a log parsing algorithm and a log detection model are proposed.The feasibility and effectiveness of the algorithm and model are verified through experiments.Combined with the actual application scenario,a log anomaly detection system based on deep learning is designed and implemented.The main research contents and results are as follows:1.A parallelization log parser algorithm based on spell(PLP-S)is proposed.This algorithm integrates the longest common subsequence log parsing strategy and parallel scheduling strategy to parse the log information in parallel structure,which improves the log parsing performance and realizes the accuracy of parsing various types of log information.Experimental results on Log Hub data sets show that the parallel structured log parsing algorithm based on the longest common subsequence is feasible,improves the efficiency and performance of log parsing,and maintains high parsing accuracy in various types of log data sets.2.A log anomaly detection model based on gated unit network(GRULog)is proposed.This model integrates the execution path exception strategy and parameter exception strategy,detects the log exception from two perspectives of log key sequence and parameter list,and determines the exception according to the actual value and predicted probability distribution threshold range.Experimental results on HDFS data set and Hive data set show that the execution path and parameter anomaly detection model based on gated unit network is effective and feasible,and has improved detection efficiency and performance compared with other log detection models of the same kind.3.Springboot and My Batis technologies are adopted to design and implement the log anomaly detection system based on deep learning.It consists of four modules:user information management module,anomaly detection engine module,task management module and anomaly alarm module.The system implements log anomaly detection calculation engine.By adding the detection model and comparison model proposed in this paper,users can select models according to individual requirements for anomaly detection and generate log anomaly detection report.This system has certain advantages in the performance and efficiency of log anomaly detection.