| Log anomaly detection is crucial for the computer system.Through analyzing and processing the logs generated by the system,the abnormal or potential problems in the system can be identified,so as to ensure the stability and reliability of the system.At present,due to the expansion of the scale and complexity of the software system,the amount of log data is far beyond the previous level,and the traditional manual detection method has been unable to meet the requirements.In recent years,the research on log anomaly detection is mainly based on deep learning combined with data mining technology.This kind of method has higher accuracy,scalability and robustness.Although the existing methods have achieved relatively good results,the effectiveness of the existing methods gradually decreases with the iterative updating of software systems and the expansion of the scale of new types of logs.Therefore,it is extremely important to develop a log anomaly detection method with high accuracy and strong generalization.In response to the issues with the current methods,the study focuses on the following aspects:(1)Detection method based on Sentence-BERT(Log SBERT): In this method,Sentence-BERT model is introduced to extract semantic behavior features of log templates,and semantic dictionary and matching algorithm of log templates are established.Anomaly detection is realized by LSTM model,and Log SBERT can effectively improve the anomaly detection performance.In the improved Log SBERTE method,the Bi-LSTM model with self-attention is used to replace the LSTM model.This method not only improves the detection accuracy,but also better solves the problem of robustness in detecting new types of logs.(2)Detection method based on parallel Bi-LSTM model(Bi SLog SBERT): Since the majority of deep learning methods are based on log templates to achieve anomaly detection,without combining other features,such as frequency statistics and embedding position,the performance of detection methods is not ideal.Therefore,Bi SLog SBERT,a parallel Bi-LSTM log anomaly detection method,is proposed in this paper,which can detect both log semantic behavior features and TF-IDF word frequency statistical features.This method can significantly improve the effectiveness of log anomaly detection methods.(3)Implementation of Log SBERT method prototype system: Py Charm integrated development environment and Py Simple GUI framework are used to realize the Log SBERT log anomaly detection method prototype system,and the system design is completed through requirements analysis.The prototype system includes five modules: Data processing,model training,anomaly detection,incremental update and detection interface. |
