Research On Botnet Communication Detection Technology Based On Machine Learning

In recent years,the smart devices and search engines that people use are all based on Internet technology.You can find that Internet technology-related products such as big data,cloud computing,and the Internet of Things are closely related to our daily lives.The network is frequently used by people,so the flow data generated by various smart devices is very large.However,some criminals obtained economic profits through Internet loopholes,resulting in a botnet.At present,botnets are extremely destructive,which not only leaks information from Internet users,but also causes the paralysis of network equipment,which brings challenges and huge difficulties to network security.Most of the current botnet detection systems can only detect one type of botnet,which makes the botnet defense work seem to be overwhelmed.In recent years,machine learning technology has been applied to all walks of life.Because it has autonomous learning ability and can imitate learning,it can adapt to many application scenarios.Applying this technology to botnet detection technology can effectively improve the detection capability of botnets.At present,some botnet detection technologies use machine learning,but there are also many defects.There are three main aspects: the use of an algorithm can only detect one type of botnet and the detection accuracy is not high.Second,the characteristics of the botnet need to be manually analyzed and obtained,which requires a lot of time and labor costs.Third,in the past,the use of Hadoop or other frameworks to build distributed systems could not meet the requirements for real-time detection of massive data.In this paper,through the study of a variety of machine learning algorithms,we explored an idea of combining different types of algorithms,and modeled the data set from multiple angles to solve the problem of a single model.The detection system consists of three detection algorithms.The first algorithm is to use the CURE algorithm to cluster the DNS traffic generated by the botnet using Fast-Flux technology and the normal generated DNS traffic characteristics;the second algorithm is to perform the traffic generated by the malicious domain name generated by the DGA algorithm After the dynamic and static feature extraction,the SVM classification algorithm is used to classify the malicious domain name features;the third algorithm is to introduce the IPFIX concept to construct the IPFIX topology,and use the improved K-means II algorithm to cluster the IPFIX topology features.In the design process,we also used the supervised learning and unsupervised learning modes of machine learning to build a botnet detection model by clustering and classifying features.This system chooses Spark Streaming to build a distributed real-time stream processing strategy platform to meet the demand for real-time detection of high-speed traffic.The design also compares this system to detect the IPFIX type botnet module and the Bot Hunter system.When the two systems detect the IPFIX type botnet at the same time,the system proposed in this paper detects the three IPFIX type botnet The efficiency is higher than the Bot Hunter system.Finally,by testing a variety of botnet families in a virtual and controllable experimental environment,the recall rate and accuracy rate of this system are higher than 81%,and the accuracy rate is 84.6%.The experimental test results show that the system in this paper meets the real-time detection requirements and can realize the detection of botnet communication of Fast-Flux,DGA and IPFIX types.