Research On Key Technology Of Botnet Detection

With the rapid development of botnet, it has posed serious threat to Internet. Thus, the botnet detection has recently become a hot research topic in the field of network security. The botnet detection can be done through the following steps: first, obtain the information which may be related with the botnet activities; second, according to the essential features represented by the corresponding information, use various analysis techniques to identify and judge the existence of botnet; finally, determine the positions of attackers, command and control servers and zombies. Although there are several research results at home and abroad, the botnet detection still has some pressing problems such as information acquisition and fusion, essential feature extraction, diagnosis of the communication and behavior, correlation analysis of detection, and system architecture.Towards the typical problems and common requirements during the botnet detection process, we make an intensive study of the current key technologies and the application deployment. And then we propose a hierarchical collaborative model and a botnet collaborative detection system based on this model. Specially, we focus on the related models and methods of botnet threat awareness and feature analysis. We also design and implement a prototype system to validate our work. The major contributions of this thesis are as following:1. Based on the analysis of the disadvantages of existing botnet detection architectures and the advantages of the collaborative work, a hierarchical collaborative (HCO) model is proposed. And then, the HCO model is designed detailedly from four levels such as model framework, data structure, modeling process and collaborative mechanism. Besides, based on the HCO model, a botnet collaborative detection system (Bot_CODS) is presented. Bot_CODS is designed from four aspects including architecture, physical structure, logical structure and working principle. The HCO model fully reflects the basic idea of botnet detection, reasonably combines with the idea of collaboration, and fully draws out the collaborative abilities of the detection from three different levels such as information, feature and decision-making. Moreover, Bot_CODS based on the HCO model has good scalability and interoperability. The detection components can be flexibly deployed in heterogeneous networks and adapt to various application environments. And the interoperation among the internal components, detection systems and other security products can be done safely and efficiently. In addition, due to the close collaborative relationship provided by the HCO model, Bot_CODS can respond quickly to the widely distributed botnet activities. Thus, Bot_CODS effectively meets the requirements of botnet detection.2. Regarding the main characteristics of botnet activities, a distributed botnet detection method based on collaboration is proposed. The botnet activities have multiple phases, various representations and wide range. Considering these characteristics, a role-based politic collaborative threat awareness model (RPCTAM) is presented. Based on the study of the existing computer supported cooperative work (CSCW), this model introduces the definition of policy, and defines the basic sets, basic relationships and the corresponding rules. According to the decompositions of roles, policies and tasks, the collaborative interoperation scope is divided. Moreover, group is used as a unit to ensure the interoperation and communication processes of inter-group and intra-group members. In this way, the collaborative efficiency and progress can be greatly improved. Furthermore, botnet activities always interfere with the security tools to make a diagnosis. In terms of this characteristic, a malicious sensor determination method based on trust measurement is proposed. Through computing the trust values of the threat awareness sensors (TASs) deployed in Bot_CODS, this method can determine whether the TAS upon the node is captured by botnet. Thus, the malicious information sent by malicious TASs can be filtered out. Using this method, the dependability of the whole system can be improved. Finally, on the support of the efficient and trust collaborative work of TASs in Bot_CODS, a collaborative botnet detection method against DDoS attack is proposed, which especially focuses on the subtle DDoS attacks launched by botnet. The key ideas of the method are listed as follows. 1) During the DDoS attack process, some traffic attributies have been changed due to the addition of malicious packets. We merge those changing traffic attributies into an indicator, called TSS (Traffic Status Snapshot). 2) The integrated deviation rates (IDRs) of TSSes during different time intervals are computed, which are helpful for the identification of suspicious attack sources (malicious IP addresses). 3) According to the synchronization of botnet attack activities, the comparison among malicious IP addresses is done by exchanging information. Then, the existing zombies can be detected from the suspicious attack sources. The collaborative detection method can reduce the false negative rate caused by the traditional methods, save the computing resources and storage space, and realize the fast and accurate detection of subtle DDoS attacks launched by botnet and zombies.3. Essential feature is the key factor which guides the botnet detection process. In order to obtain the effective essential features, a botnet feature extraction method towards command and control (C&C) is proposed. Regarding the representation and relationships of botnet essential features including signature, anomaly, character pattern, a botnet feature description method is presented. Then, the detailed content of feature information is defined, and the abstract description of feature information is done by use of BNF (Backus-Naur Form). Besides, a description language based on XML called FIDL (Feature Information Description Language) is also defined. In FIDL, the feature information is described as the document format with unified structure used for the TAS. Thus, the efficiency and flexibility of detection work can be improved. In addition, C&C channel must pass through the network, and the attack commands always have a relatively fixed format and command strings. Thus, a signature generation model against C&C channel is proposed. The model is composed by five parts including pre-filtering, protocol classification, data preprocessing, signature generation and determination. Moreover, according to the differences of the response modes to the attack commands for bots, a determination method targeted at the attack commands is presented. This model can be applied to the network traffic of border networks, and mainly solve the problem of weak applicability of the honeypot or honeynet. It can generate the signatures with the command format from the botnet communication accurately. It also can integrate several signature generation techniques including the multi-sequence alignment algorithm used in this thesis, and meet the requirement of botnet feature extraction towards C&C.4. Toward the botnet characteristics such as rapid expansion, great and instant harm, a botnet feature fusion method based on PHT (Prefix Hash Tree) is proposed, which includes two aspects: feature aggregation and feature access. In Bot_CODS, all of the threat monitor centers (TMCs) aggregate the local feature information step by step through the platform which is built based on PHT. And then, according to the aggregation rules, the global information is formed by the gathered information and stored distributedly into the characteristics library. In this way, the local feature information can be comfirmed at the shortest time in the whole network, and the corresponding TMCs in Bot_CODS can prepare well for the coming task. Moreover, a feature information access algorithm based on PHT (FIA-PHT) is presented. According to the naming and distribution process, TASs use the multi-attribute range query method. Thus, the feature information of the feature library can be inquired and accessed quickly. The more specified detection ability can be assured for TASs under the TMC. By doing the theory analysis and the simulation based on real traffic, the accuracy and feasibility of this method is proved. Experimental results indicate that it is significantly better than the same kind of solutions in the aspects of request and access latency and node load.5. Based on the study of the key technologies described upon, a Bot_CODS prototype system is designed and implemented. We specify the design details of key equipments including TAS, TMC and TDC (Threat Decision-making Center), etc. The Bot_CODS prototype system integrates some softwares and tools including topology discovery, traffic capture and intrusion detection. And the distributed botnet detection method based on collaboration idea, botnet feature extraction method towards command and control, and botnet feature fusion method based on PHT are all implemented in the prototype system. According to the proposed testing content, the model correctness of HCO is also validated.To sum up, our research is a beneficial exploration of botnet detection. It has the good theoretical and practical value to the development of botnet detection. The research has been integrated into the national high-tech research and development plan of China, the natural science foundation of China and our actual project.