Multidimensional And Adaptive Anomaly Detection System Of Web Attacks

Through the data mining and machine learning techniques, behavior patterns of web users can be extracted from massive online data. In this way, not only the heavy workload brought by manual analysis can be reduced, but also the difficulty of secure rule matching can be decreased. However, in recent years,the huge amount of dynamic HTTP requests have raised great challenges to traditional detection system in web applications, such as high false positive rate,poor adaptability, easy over-fitting, and high time complexity. Moreover, with the increasing of the diversity of computer systems and network environments,the entry points and behavior patterns of malicious attack are constantly evolving. All these limitations make the traditional anomaly detection system inappropriate to the enterprise-level detection requirements.In this paper, we propose a novel anomaly detection system, with the characteristics of high performance, low latency, multi-dimension and adaptability. The contribution of this paper are five folds. (1) Statistical model construction: We first extract the web access data under each specific domain name, calculate the statistical characteristics parameters of the specified detection characteristics under specific domain names, and construct the corresponding statistical model. (2) Multidimensional subsystems construction:According to the calculated statistical characteristic parameters, the behavior characteristics of each record in the web access data set are detected from different dimensions, and establishing corresponding eigenvectors. (3)Classification model construction: Based on the multidimensional eigenvectors and the actual class label generated by the statistical model, we adopt the state-of-the-art classification algorithm to detect the records of abnormal data. (4)Model fusion: In order to improve the detection performance of the system, we further fuse the models of different classifiers. The model after fusion always achieves better performance than any previous single model. (5) Results feedback: It is necessary to judge whether the current statistical model and the classifier are invalid according to the statistical characteristics of the detection results. If it fails, the statistical model and classifier need to be rebuilt automatically.The detection performance of seven single classifiers and fusion models are evaluated based on the massive web access logs provided by Qihoo 360 Company. The experimental results demonstrate that our proposed system achieves substantial improvement over the previous single model and other classical detection models.