Research And Implementation Of DDoS Attack Detection Technology For Hadoop

As the most widely used open source distributed system architecture,Hadoop is widely used in various fields such as digital currency,electronic cash,network insurance,and smart medical care.Its security is also receiving more and more attention.Hadoop was not designed with security in mind at first,and the default cluster is not attacked by malicious nodes.With the increasing complexity of the network environment and the real?time requirements of cluster computing,Hadoop has become more threatened by attacks such as external scanning,penetration,denial of service,and worms.DDoS sends a large number of legitimate requests containing false information to the network through multiple hosts,occupies the normal user's access bandwidth and achieves the effect of denial of service.It is a non-negligible attack in the network security environment.This paper focuses on the DDoS attacks that Hadoop may suffer.Based on the repeated study of a large number of domestic and foreign literatures,this paper analyzes the DDoS attack methods against Hadoop from Hadoop architecture and security mechanism,and proposes a DDoS based on GA-SVM.Attack detection method.The main work and results are as follows:1.DDoS attack experiments on Hadoop clusters with or without HA mechanism and Zookeeper configuration.This paper analyzes the impact of Hadoop's different deployment methods and security configurations on the response mechanism of DDoS attacks.It summarizes the most effective attack methods for Hadoop,and combines this attack to develop DDoS attack detection technology for Hadoop.2.The DDoS attack detection is performed by using the entropy-valued attack traffic feature sequence.Information entropy has a fast and accurate measurement of the uncertainty of variables.For different types of DDoS attacks,traffic characteristics are selected to select easy-to-obtain and representative header information such as IP and port.A fixed number of consecutive data packets are used as computing units,and information entropy and condition entropy of traffic characteristics are used as algorithm input features.Sequence,training for DDoS attack detection model.3.Use the"one pair of remaining"SVM algorithm for DDoS attack detection and optimize its parameters.The effect of the SVM algorithm training model depends on certain parameters,and most of these parameters are set by experience.The optimization algorithm can improve the accuracy of parameter setting.GA(genetic algorithm,genetic algorithm)is used to optimize the scale parameters and penalty parameters in SVM algorithm,and the SVM algorithm based on GA optimization is designed and implemented.The algorithm can find the optimal SVM initial parameters in the global search space.The experimental results show that the algorithm can effectively improve the detection accuracy of the initial S VM algorithm.4.Perform flexible alliance organization on multiple nodes of the NameNode node group.For Hadoop DDoS attacks,the target is multiple host nodes.Attack detection also requires the organization of multiple detection points.Multiple detection points in the alliance can realize real?time communication of dynamic detection and detection frequency,and the control node comprehensively judges the detection effect,thereby analyzing the state of the entire cluster subjected to DDoS attacks,distinguishing the degree of attack,and performing organized defense.5.Designed and implemented a DDoS attack detection system for Hadoop.According to the Hadoop DDoS attack detection process,the data acquisition,file import,data preprocessing,model analysis,and result display in the detection process are modularized,and the system functions are tested.Experiments show that the proposed DDoS attack detection system for Hadoop can detect common DDoS attacks,and the attack recognition rate is over 90%,which has a good detection effect.