Research On Feature Selection Algorithm And Its Application In Anomaly Detection

As the progress of society and the development of Internet and information technology, the Internet penetration rate and the number of Internet users have increased steadily, and people’s daily life and work are inseparable from the network. At the same time, network attack means and tools become increasingly diverse and complex, and network security is facing serious challenges. Intrusion detection is an important security technology. It collects and analyzes data from netwok, and then determines whether there are intrusions in the network. It takes appropriate measures when intrusions are detected. However, as the scale of the network and the number of users are increasing, the amount of data transmitted over the network is to a "explosive" trend, which makes intrusion detection systems can not handle large amounts of information in real time, and results in the response of IDS is not timely or even failure.In order to solve the above problems, the researchers turn their attention to feature selection methods. It can preprocess the data before it is input into intrusion detection systems. Feature selection methods select features which are important to intrusion detection systems, and reduce the dimension of data, thereby improve the efficiency of intrusion detection systems effectively.Overall, the main contribution and the specific contents of this thesis includes the following aspects:(1) Research on concept, models and other related theoretical knowledge of intrusion detection. Study the classification of intrusion detection methods, analyze and compare the advantages and disadvantages of different intrusion detection methods. Propose an improved TCM-KNN anomaly detection algorithm(ITCM-KNN), and apply it to Do S anomaly detection model. Summary current problems of intrusion detection systems.(2) Research on feature selection algorithms, including filter model, wrapper model and hybrid model, analyze and compare their advantages and disadvantages. Focus on researching several typical feature selection algorithm, including Correlated Feature Selection, Info Gain, Info Gain Ratio, Relief and Chi Square, analyze principle of each algorithm, compare their advantages and disadvantages.(3) Based on the above studies, this article proposed an effective feature selection approach based on bayesian network classifier. This approach selected subset of features that is conducive to distinguish normal and abnormal data, while retain high detection rates and low false positive rates. After feature selection, irrelevant features and redundant features are removed, thus time and space costs of anomaly detection are decreased, and detection efficiency is improved. The proposed approach is used in anomaly detection model. To verify the effectiveness of the approach, we designed several experiments based on NSL-KDD benchmark dataset. We compared the experiment results with those methods in(2) from time costs, true positive rates, false positive rates and the accuracy of classification.