Research And Implementation Of Anomaly Detection Method For System Log

The constant advancement of computer technology has offered mankind convenience and progress,but it has also brought significant obstacles and risks.Vulnerabilities in the computer system itself will cause issues with the system’s functionality.Furthermore,some people with ulterior purposes may attack the computer system,leading it to malfunction.The system log is a type of file that records the computer system’s operation status.By analyzing the log,system operation and maintenance employees can discover the fault,conduct an in-depth investigation of the problem and resolve it.As a result,log anomaly detection is critical for computer system security.So far,researchers have proposed some anomaly detection methods for the problem of log anomaly detection,but there are still some flaws,such as incorrect classification in log parsing,ignoring the detection of parameter variables during anomaly detection,and producing poor detection results for long sequences.Based on the foregoing,this thesis seeks to investigate and design log parsing methods and log anomaly detection models,as well as to implement a viable log anomaly detection system.The following are the specific research and innovation points:(1)To address the issue that existing log parsing algorithms may incorrectly classify logs during the parsing process,a log parsing method based on parsing tree PTCC(Parsing tree combine clustering)is proposed.The problem of improper log template classification is improved by calculating the similarity of log strings and setting suitable thresholds for aggregation.Simultaneously,log parsing trees are built utilizing parsing trees,which not only improves accuracy but also increases parsing performance.Finally,studies using publicly available datasets demonstrate the usefulness of the suggested algorithm.(2)An anomaly detection model LTCV(Log template combine variable)combining log templates and variables is presented to address the difficulties of poor long-sequence detection and disregarding the detection of parameter variables during log anomaly identification.The model detects anomalies not only on log sequences,but also on log variable meanings.The sparse attention technique is used in sequence anomaly detection to acquire not only the global features of the sequence,but also to tackle the problem of poor detection effect on large sequence logs.Simultaneously,the detection of log variables is incorporated to improve the model’s accuracy.Finally,a set of experiments demonstrate the efficacy of the suggested algorithm.(3)A log anomaly detection system is created based on the log parsing method and log anomaly detection model given above.The system combines the two techniques mentioned above,provides a high-precision log anomaly detection capability,and visually displays the findings of anomaly identification to users.