Research Of Outlier Mining Algorithms Based On Space Partitioning In High-dimension

With the fast development of technology of satellite remote-sensing, data analysis, Virus defence, and so on, a study of detection technology for finding the new types of intrusions is one of urgent problems which should be solved in recent years. To solve the problem of'dimension disaster'in high dimensional space, this paper focuses on the space partitioning-based outlier mining algorithms in high-dimension, and it is used for intrusion detection to discover the intrusions, which are viewed as'small pattern'events. The subject selection has important significance in theory and practice for establishing the efficient intrusion detection model.Firstly, an improved outlier detection method in high-dimension based on weighted hypergraph is presented. The data set is partitioned. To reflect comprehensive the data pattern of data set, the attribute similarity is defined to obtain the weight of hyperedge in data set hypergraph model. The meaningful outliers in high-dimension can be mined by means of appropriate user-defined threshold.Secondly, a novel algorithm based on single-parameter-k local density in high-dimension for mining outliers is proposed. The data set is divided into some clusters. As long as the distance-independent parameter is input, and the analysis of the data distribution in the object p's k-neighborhood, p's suitability for a core object is judged. Experimental results show that our algorithm gained better clustering quality and lower memory usage and time cost than DBSCAN and LOF.Thirdly, a method locally linear embedding based on optimize survey distance for mining outlier in high-dimension is presented. In order to show correlation among all data, graph theory is utilized to construct the data structure. To optimize the K neighborhoods structure of each object, the neighborhoods of object are established according to the concept of new survey distance. Our algorithm obtained better dimensional sensitivity than the research of detection of outliers based on locally linear weighted value.Lastly, an intrusion detection method based on mining outliers'feature attribute sets is proposed. Outlier sets are obtained by an outlier detection algorithm, it is based on improved Euclidian distance, and the corresponding feature attribute sets are mined. Each outlier and its feature attribute sets corresponds to one attack signature. Furthermore, misuse intrusion detection rule database is consisted. Experimental results show that the efficiency of this method is improved.We implement the above algorithms with language of VC++. According to the contrast and analysis of different processing results, the algorithms proposed in this paper are more efficient than the current ones,and the anticipated results are realized.