Detecting stealthy scans and scanning patterns using threshold random walk

Scanning is a precursor to many intrusions and attacks. In the absence of insider or public information about a target network, scanning is the first step in obtaining basic information about the target network. Detecting these initial scans may allow defenders to block potential attackers before they learn enough to be dangerous. The threshold random walk (TRW) is one of the most effective algorithms for early scan detection. It uses sequential hypothesis testing to distinguish between a knowledgeable user who occasionally sends misaddressed traffic and an intruder randomly probing for targets. Many network intrusion detection systems have integrated TRW scan detection e.g. Bro, Snort and "rwscan" API in the SiLK Tools suite.; One of the problems faced when using TRW is detecting stealthy or slow scans. TRW is a threshold based algorithm. It has two thresholds defined in terms of the proportion of unsuccessful connection attempts by the subject. The higher threshold, when crossed, favors the hypothesis that the subject is a scanner. The lower threshold, when crossed, favors the hypothesis that the subject is benign. If the scanning is slow or stealthy, the machine does not reach either of the thresholds rapidly. Techniques to defeat TRW involving repeated connections from the subject to "known good" targets have been proposed. At any time, the state of the system can be described by disjoint sets identifying subjects classified as scanners, as benign, or as undetermined. We investigate a novel method using a Bloom filter to restrict the input to TRW. The Bloom filter also allows flexibility in the definition of a target as it can pass unique source and destination IP pairs or can include ports in its consideration. The analysis done on a month of the trace has shown that using the bloom filter approach reduces the false positives as compared to TRW. Using the classification of the hosts based on the output of the TRW, we are able to identify unique behaviors and patterns of the stealthy scanning hosts.