Design And Implementation Of Log Collection And Analysis System Based On Web Application

With the rapid development of Internet technology, network has penetrated into all aspects of people’s lives. As the entrance of Internet information service, Web services provide an open platform for massive information resources. The openness of Web services makes Web application attacks increase significantly. Web log records user behaviors for the entire Website visit. Hence, security analysis on Web log can not only reconstruct attack scenarios but also monitor the state of Web server in real time. Currently, Web attacks are so complex that the single use of rule base based misuse detection method is unable to cope with the emerging attacks. Motivated by this, this dissertation designs and implements a hybrid Web log security analysis model combined with misuse detection and anomaly detection. The main works of this dissertation can be summarized as follows:(1) A feature extraction method for Web log request attribute is studied. The request attribute of Web log contains the most attack characteristics. The request feature vectors are extracted to distinguish abnormal behaviors. And then, a Web normal access model based on request feature vectors is established by using K-means clustering algorithm.(2) A hybrid Web log security analysis model is proposed, which fully combines the advantages of the misuse detection model based on rule base and the anomaly detection model based on clustering algorithm. The malicious logs, which cannot be detected by misuse detection model, will be loaded into anomaly detection model for the second detection. The test data indicate that compared with single attack detection model, the hybrid model proposed can effectively improve the detection rate and reduce the false alarm rate.(3) A log collection, storage and display platform based on ELK stack is built to achieve massive Web log attack detection in real-time. The test data show that ELK stack has good performance on the data loading rate and the query response delay for massive data.