Research On Log-based Anomaly Detection

With the rapid development of Internet technology,the network is more closely related to people's daily life,and the problem of network information security has attracted more and more attention.In recent years,frequent network security incidents have brought many bad effects to people's lives.The log records the detailed information during the running of the system.By analyzing the logs,you can detect the abnormality of the system and find out the attack behavior,so as to protect the user's information security from the source.This paper mainly studies the anomaly detection method based on log data.The main innovative work has the following two points:(1)Anomaly detection method based on convolutional neural network.In order to solve the problem of low accuracy and false positive rate of log data anomaly detection results,this paper proposes an anomaly detection model based on convolutional neural network.First,the original log data is preprocessed,then the one-dimensional log data is mapped into two-dimensional image data,and then the feature is extracted from the convolutional neural network for abnormality detection.Experiments show that the convolutional neural network is introduced to detect the abnormality of the log data,which improves the detection accuracy of the detection results compared with the traditional machine learning anomaly detection method.(2)Anomaly detection method based on fuzzy kernel clustering and active learningIn order to reduce the problem that the cost of obtaining anomaly detection for log data is too high,this paper proposes an algorithm based on fuzzy kernel clustering and active learning.Firstly,the data is pre-processed,then the fuzzy kernel clustering algorithm is used to cluster the candidate samples in high-dimensional space,filter the redundant points of the samples,select the cluster center to mark the initial classifier,and finally combine active learning with The small mark cost optimizes the anomaly detection model.The experimental results show that the proposed method can obtain the performance improvement of the anomaly detection model by using fewer labeled samples.