Design And Implementation Of IDS Running Anomaly Detection System Based On Log Similarity

With the rapid development of the Internet,while the network has brought great convenience to people,the challenges that network security faces are also getting tougher.As an significant component of network security,Intrusion Detection Systems provide important services such as network monitoring and network anomaly detection,which play an important role in network security.Taking the running security of the Intrusion Detection Systems as the goal,this paper took the log data outputed by the IDS as the research object and conducted anomaly detection research on the running status of the IDSs.This paper designed and implemented an IDS Running Anomaly Detection System based on log similarity.This system conducted anomaly detection research based on IDS output logs,detected outliers in log data,and then analyzed whether the IDS equipment itself was running abnormally.At the same time,the system provided personalized queries of historical logs and real-time logs and Web interface display,and can display the results of anomaly detection for observation and further analysis.The system included four main parts that are log collection,log storage,anomaly detection and result display.During the system construction process,the following key technologies were resolved:(1)Extraction of pattern of IDS traffic log under high-speed output environment of IDS.To avoid packet loss and performance overhead caused by the traditional CPU-based interruption method,the system used DPDK to collect real-time traffic logs in the high-speed output environment of the IDS device and performed pattern extraction.First,initialized the DPDK working environment by configuring the UIO mechanism and 1GB hugepages.Then,started the traffic collection thread and pattern extraction thread and exclusively use the CPU core to perform the traffic collection and pattern extraction respectively based on CPU affinity,and saved the extracted log pattern data to MySQL.Based on the above,the system implemented the pattern extraction and storage under high-speed output of traffic logs of IDS devices.(2)Real-time running anomaly detection of IDS itself.Due to the similarity of the traffic processed by IDS,the running logs generated by IDS when processing the traffic show a certain regularity.Based on this,the paper used the Prophet model to fit the log data,constructed a prediction model to predict the change trend of the log data,and compared the real value with the prediction confidence interval to detect outliers in the log data.Then analyzed whether the running status of the IDS equipment itself was abnormal or not.Experiments showed that this method can effectively detect outliers in log data.This method can not only perform anomaly detection on real-time log data and detect outliers in log data in real time,but also on historical logs.The experimental results showed that the system can meet the requirements of traffic collection and storage under high-speed environment,and can make personalized query and anomaly detection of historical logs and the real-time logs as well as display of anomaly detection results,which facilitated users to understand and further analyze the running status of the IDS device itself.