Research On Network Intrusion Detection Based On Benford's Law And Machine Learning

The popularity of the Internet has brought enormous security risks while benefiting people.The escalating network intrusions may lead to a series of major security issues such as personal privacy leakage and system failure.Up to now,intrusion detection technology has been improving day by day.The use of new technologies such as machine learning solves the problems of rigidity and poor adaptability in traditional intrusion detection,and at the same time,improves the detection rate to a certain extent.However,due to the limitations of machine learning algorithms,the existing solutions still face two major problems: first,how to define the normal region containing all normal behaviors in the changing network environment;Second,how to quickly and efficiently identify intrusion behavior from massive network flows.In addition,the problem to be solved is not only the improvement of detection technology,but also the actual deployment of intrusion detection system in the real world.Most of the existing studies only focus on complicated models using several algorithms to improve the detection rate,but seldom consider the actual consumption of computing resources and computing capacity constraints.Facing the above problems,this thesis proposes a layered intrusion detection framework Filter-XGBoost combining Benford's law with machine learning algorithm.The major research contents include:(1)Aiming at the problem that the traditional anomaly detection schemes fail to define the normal network traffic profile,a general rule for distinguishing between normal and abnormal traffic using Benford's law are proposed and applied to construct a detection model based on fixed threshold.Considering that static threshold lacks flexibility and may not be applicable to different network scenarios,an adaptive threshold detection scheme based on CUSUM and EWMA is proposed at the same time.The experimental results show that both schemes have good recognition ability for normal network traffic,while the adaptive threshold scheme has higher flexibility and detection rate than the fixed threshold detection scheme.Besides,the scheme can shorten the detection time by detecting multiple flows at a time,thus effectively dealing with the input of massive data.(2)Aiming at the problem that only the coarse-grained detection at the window level can be realized using single threshold-based detection scheme,we suggest using it as an early warning system combing with other IDS.Considering the good scalability and powerful performance of XGBoost,this thesis proposes a layered hybrid intrusion detection framework Filter-XGBoost.The first layer of the detection framework is an adaptive threshold-based detection model,and the second layer is a Bayesian optimization algorithm(BOA)-based XGBoost detection model to further analyze the attack window detected in the first layer to achieve fine-grained detection accurate to a single flow.Compared with the separate detection models,Filter-XGBoost combines the advantages of both detection models.Compared with other algorithms,Filter-XGBoost performs well in detection rate and false alarm rate.