Research And Implementation Of Bot Detection Based On API Hook Technology

With the prevalence of network, Botnet has become one of the most serious security threats in modern society, which makes detecting and defending against Botnet an urgent and essential research task for the security institutions. Most of the research in Botnet detection is based on the analysis of group activities and patterns. Methods of single Bot detection are very rare. Bot detection based on API HOOK technology is a brand new attempt by a group of people leading by Al-Hammadi. Although their methods are effective, the research on Bot Detection is still limited.In order to counter against the threat of Botnet, based on the method used in "Detecting Bots Based on Keylogging Activities" proposed by Al-Hammadi and Aickelin, we make an improvement of the original method of correlation counting and add new behavior characteristic-windows monitoring to the method, which makes the detection rate of Bots increased a lot. And, we study in detail other activities besides keylogging, correlate different behaviors and put forward a new Bot detection method on Windows system-BDA. The main steps of BDA are as follows:first, it calculates the subordination degree of unknown process's different behaviors and integrates subordination degrees to form the fuzzy set of unknown process; second, it uses Grid Lattice degree to correlate the fuzzy set and the fuzzy sets of known processes; third, it distinguishes the type of unknown process by F Pattern Identification. Experiment results show that BDA can detect Bots with a high detection rate and can well distinguish between normal processes and Bot process with a low false positive degree. At last, we analyze the functional requirements of BotDetector prototype system, design two kinds of API HOOK algorithms which realize API HOOK of all the suspicious processes in the system including running process and new created process, and implement BotDetector prototype system.