Study On Online Botnet Detection Based On Network Traffic Analysis

Network security has concerned a lot of attention due to the development of Internet technique. In the recent past years, botnets have become the most common attacking tools which bring a lot of problems for network security administrators. The main attacking and spread techniques of botnets has been improved a lot since new emerging bot programs will always add new functionality to advance with the time. As one of the most serious threats of Internet, botnets have become a huge and widespread network with complex malicious functions.Currently some researchers have paid lots of attention on botnet detection and proposed a series of techniques to search botnet hosts. However, most of work just focused on the offline detection strategies which could not meet the requirements of efficient botnet detection. Meanwhile, most of these techniques paid more attention to detecting the IRC botnets rather than the P2P botnets. None of them has considered the general online detection technique for all types of botnes. According to the above situations, we study on the online botnet detection techniques in this dissertation, which focus on the detecting efficiency as well as accuracy and propose detecting strategies for the IRC and P2P botnets. Furthermore, we propose a general detecting strategy for all types of botnets.This dissertation studies some key techniques of the online botnet detection. The main contributions are listed as follows:(1) A data stream model construction technique for the network traffic is proposed. Original network traffic data would be transformed into the data stream model. And different type data streams of network traffic would be created to meet the detecting requirements in different situations.(2) An online detection method based on high similarity search is introduced to search the IRC botnets. Firstly, we build the feature streams by monitoring the network traffic. Then we search the similarities by the incremental Discrete Fourier Transform(DFT) among huge feature streams and hunt for the IRC botnet traffic. Finally, the IRC botnet structure would be constructed by topological analysis.(3) A periodicity and similarity analysis based detection technique is given to probe the P2P botnets. Network traffic would be composed into communication streams in the first step. After that, a Boolean auto-correlation method is adopted to find the periodical pattern and an incremental clustering method is used to seek the similarities among communication streams. In this way, we could pick out all the suspected P2P botnet whose identification would be confirmed if they have malicious behaviors by actively monitoring.(4) A novel online detection technique based on host behavior anomaly is introduced for latent P2P botnets. We firstly construct the host behavior stream and extract the typical behavior feature. Then suspected P2P botnet host would be distinguished when analyzing the anomalous behaviors by hypothesis testing technique. We search the abnormal situation by monitoring the entropy of network traffic and adopt a pre-processing strategy where the detecting engine is shutdown when there are no anomalies. In this way, the process efficiency would be greatly improved.(5) A protocol-independent online botnet detection technique is proposed. Firstly, we distinguish the network traffic with botnet protocols by the incremental classification method which partitions network traffic into different groups according to the protocol type. We revise the decision rules in the classification process to meet the requirement for further botnet search. After that, we adopt the dynamical clustering technique to accomplish the general detection for the botnet host. The clustering process would be adjusted in a data-adaptive way to avoid huge repeated clustering operations and reduce the computation cost. All types of suspected botnets could be detected rapidly from huge network traffic.In summary, this dissertation dedicates to study the online botnet detection technique based on network traffic analysis. Experimental evaluations show that these methods could meet the demand sufficiently for online botnet detection and have great advantages in detection precision and efficiency. We hope that these approaches and techniques could make some referential values in developing high-performance online botnet detection system.