Research Of Host-based Botnet Detection

A bot is a host that has been compromised by malware under the control of a botmasterthrough Command and Control (C&C) channel (i.e., IRC, HTTP, P2P). When many botswork together, they form a botnet. The botmaster can utilize botnets to conduct various cybercrimes, such as spreading malware, conducting DDoS attacks, spamming, and phishing.Recently, botnets have become the major platform for most online criminal activities.To solve these problems, we propose a multi-feedback approach, BotCatch, to detectbots effectively and efficiently on a host by leverage of a combination of signature andbehavior. BotCatch includes five modules: an analysis engine, a signature-analysis module, abehavior-analysis module, a correlation engine, and a multi-feedback module. The analysisengine assigns the suspicious file to either the signature or behavior-analysis module. Thesemodules analyze the file and generate detection results for the correlation engine. Thencorrelation engine correlates signature and behavior detection results to generate the finaldetection result. The multi-feedback module uses the signature, behavior, and correlationresults to dynamically adjust BotCatch. It optimizes the signature-analysis module bymaintaining the signature database. It optimizes the behavior-analysis module by maintainingthe sample set and guiding the module’s learning procedure. It optimizes the correlationengine by modifying the parameters. Our evaluation results show that (1) The correlationalgorithm in BotCatch is efficient and effective in combining signature and behavior detectionresults.(2) The multi-feedback mechanism makes BotCatch adaptive to samples andgradually becomes more robust and accurate.(3) Other correlation algorithms, such as thoseof support-vector-machine (SVM) models, are also effective; however, our correlationalgorithm with its multi-feedback mechanism provides better detection results.Our work makes the following contributions:(1) We identify two specific features ofmultiprocess bot: separating C&C connection from malicious behaviors, and assigningmalicious behaviors to several processes. Then we theoretically analyze why existingbehavior-based bot detection approaches are less effective with multiprocess bot according tofour categories of behavior-based approaches.(2) We present two critical challenges ofimplementing multiprocess bot, and implement a single process and multiprocess bot from asimplified version of Zeus. We use signature and behavior based detection approaches toevaluate them. The results indicate that multiprocess bot can effectively decrease the detectionprobability. Then we propose other multiprocess bot architectures and extension rules, andexpect they can cover most situations. In this paper, we plan to design more robust features and correlation mechanisms todetect evolving social bots through in-depth analysis of the evasion mechanisms utilized bycurrent social bots. To achieve our research goals, we collect most of existing social botnetsource codes or builders and collect their execution traces. Then, we analyze existing evasionmechanisms of social bots including four basic evasion mechanisms and four advancedevasion mechanisms. We validate these mechanisms by examining three state-of-the-artdetection approaches on our collected traces. Based on the in-depth analysis of those evasionmechanisms, we design nine new features classified into life cycle-based and failure-basedand two correlation mechanisms, temporal correlation and spatial correlation mechanisms todetect social bots. Temporal correlation accumulates the historical information to fight againstthe advanced evasion mechanism, delayed response, and spatial correlation combines therelationships between different processes to fight against multiple processes. Through ourevaluation, we show that our newly designed features and correlation mechanisms areeffective to detect evolving social bots. Our approach with classifier random forest getssignificant results about0.3%FP rate,4.7%FN rate,0.963F-Measure value and99.2%detection rate.