Study On Ensemble Learning And Its Application In Intrusion Detection

Ensemble learning that trains multiple component classifiers and combines their predictions has evolved remarkably in machine learning communities. This technique can improve generalization ability of learning system significantly and has been regarded is an effective and efficient engineering method for intelligent computing. In the thesis, neural network ensemble, ensemble learning based data mining and ensemble learning applied in intrusion detection are studied, and some innovative contributions are achieved as follows:1. A novel selective approach to neural network ensemble named FASEN is presented. After every neural network is trained separately, FASEN selects those has low estimative generalization error and high dissimilarity with others according to results tested in validation set, then ensembles them. Theoretical analysis and experimental results show that FASEN is better in efficiency and generalization than other approaches.2. An unsupervised feature selection approach which utilizes multiple classifiers ensemble technique is proposed. As clustering results generated by some algorithms are usually different from each other, feature selection performs multiply and all results are combined to produce final selected features. In addition, ReliefF is ameliorated and employed as an essential part in the approach. Experimental results indicate that the approach can remove redundant features and improve quality of clustering.3. Based on ensemble learning, a distributed clustering model is proposed. As an implementation for the model, a novel distributed K-means called DK-means is presented. Despite data distribution varies in any local site, DK-means always works well. Experimental results show that DK-means can achieve similar results to centralized clustering.4. A novel distributed intrusion detection method based on neural network ensemble is proposed. The distributed detection is implemented by a ranked ensemble algorithm, which firstly does detection in single agent with an ensemble of neural networks and secondly cooperates with other agents to obtain detected outcome while one agent can't detect by itself. When discovering a new kind of attack, neural network ensemble is updated by a RAN (Resource Allocating Network) based incremental learning algorithm.5. An actual intrusion detection system is implemented with JAVA language. Experimental results on the system show that the presented algorithms are effective and efficient in detecting attacks.