Research And Implementation Of Host-based Bot Detection Technique

Botnet has become one of the most serious security threats in modern society.Detecting and defending against Botnet has become an urgent and essential task for the security institutions. Most research in botnet detection is based on the analysis of group activities and patterns. However, the activities or pattern can not be identified if the botnet’s entire structure is unknown. Host-based detection is a brand new attempt.Unfortunately, current approaches are few and suffer from ineffective detection models.So new approaches to dectect Bot at end host are urgently needed.In this paper, a novel botnet detection method at the end host is proposed. Invocation characters of API functions when Rxbot respond to the commands are analysed with API HOOK technique firstly.These characters are used to conclude the invariant characteristic of current Bots and then the detection algorithm is presented. The approach monitors the processes at the host and filters the processes with whitelist.The processes which not in the whitelst and run when the system starts will be labeled as suspected processes.Then the PE files of these suspected processes will be used to extract signatures, if any of these signatures equal to the signatures of known Bots, detection system will alarm.Other suspected processes will be injected a dll file to get the API functions sequence. The fast detection approach and the sequence detection approcah dectect abnormal invocation to API functions cooperatively. The former method relies on the link between the arguments used in some important API functions and commands issued by Botmaster while the latter relies on the differences between Bots’and normal softwares’API functions invocation sequences and then SVM is brought to identify if an unknown sequence is generated by a Bot.Series of experiments show that this approach can effectively detect running bot on an end user’s host.