Research On Outlier Mining Algorithms Based On Neighborhood Relation

With the continuous development of computer networks, cyber security issueshave been increasingly outstanding. The traditional static cyber security technologylike Firewall and Data Encryption is not able to fully meet user demands. As animportant dynamic security technology, NAD (network anomaly detection) has beendrawing growing attention from researchers and users, and has become a salientresearch topic. Existing methods for NAD have generic problems such as high FP(false positive) rate and unsatisfactory TP (true positive) rate, etc. To address theseproblems, using the granular computing neighborhood model, this thesis mainlyfocuses on researching the outlier mining method based on neighborhood relationand its extension to NAD. The main research is conducted as follows:1. A Neighborhood Entropy based outlier detection algorithm named NROD(Neighborhood Relation-Based Outlier Detection) is proposed, which definesthe neighborhood partition to form the neighborhood granule. It leverages theadvantage of RNE (Relative Neighborhood Entropy) at measuring theuncertainty between data objects to determine the strangeness of every object,and thus perform the outlier mining more accurately and effectively.2. The concept of RNE in the NROD is introduced to the TCM (Transductiveconfidence machines) framework, and the RNE is leveraged as a new tool tomeasure the degree of anomaly, which re-defines the Strangeness. An algorithmbased on RNE, named TCM-RNE (Relative Neighborhood Entropy), isproposed.3. Aiming at the NROD and TCM-RNE algorithms, experimental verification andanalysis have been done based on UCI and KDD Cup1999datasets,respectively. Experiments show that NROD is effective and outperforms thetraditional algorithms when dealing with continuous or mixed dataset. On theother hand, compared with the TCM-KNN (K-nearest neighbours) proposed byLi et al., when the ratio of outlier data to the whole data is between1%-2%, the TCM-RNE has a slightly better TP under specific attacks (e.g., U2R), but the FPof TCM-RNE is moderately decreased in all scenarios, by7%in average.Results also show that the TCM-RNE performs excellently under "noisy"environment.