Research On Network Intrusion Detection Based On Machine Learning

With the widespread use of computers and networks around the world,the complex network structure promotes the generation of new threats,and the pattern of network attacks has increased significantly,intrusion detection systems and related technologies have obtained a great deal of attention from the industry as an effective means of defense.The main research results of this thesis are as follows:(1)For the current Internet environment,there are two main challenges to the detection of attacks.First,network traffic is large and produces rapidly.Processing these large volumes of network data requires more effective data analysis techniques,and deep learning is an efficient data analysis technology widely used whthin various domains.Second,the feature selection problem,the variety of attacks pattern and the suitability of different scenarios.Formerly,when machine learning was used for real tasks,the features of describing samples were usually designed by domain experts,and specialized knowledge was needed to process data.Traditional methods do not meet requirement of the network in the aspects of adaptability and efficiency.Getting some resolution of the problem above mentioned is necessary.A machine learning method was proposed for NIDS,by which features can be learned and adjusted itself according to previously undefined attacks.In particular,the improved autoencoder to learn the features of the input data and extract the key features.Soft-max regression classification was applied at last.Using NSL-KDD intrusion detection benchmark dataset was evaluated to prove the usability of model proposed in this paper.This simple network architecture,as the result,can accurately predict a large amount of data in a short time.(2)In the modern high-speed Internet environment,traditional machine learning-based intrusion detection often only considers a single algorithm to identify intrusion data,lack of the flexibility method,low detection rate,no handing high-dimensional data,and cannot solve these problems well.In order to improve the overall performance of intrusion detection system,a novel general intrusion detection framework was proposed based on previous research,which consists of five parts:preprocessing module,autoencoder module,database module,classification module,and feedback module.The data processed by the preprocessing module are compressed by the autoencoder module to obtain a lower-dimensional reconstruction feature,this low-dimensional reconstruction feature can effectively improve the accuracy of the classifier,and the classification result is obtained quickly through the classification module.Compressed features of each traffic are stored in the database module which can both provide retraining and testing for the classification module and restore these features to the original traffic for postevent analysis and forensics.These five parts constitute the three main functions of the framework and complete the whole framework.For evaluation of the framework performance proposed,simulation was conducted with the CICIDS2017 dataset to the real traffic of the network.As the experimental results,the accuracy of binary classification and multiclass classification is better than previous work,a comparative analysis the training and testing time of the algorithm under different parameters to ensure that the overall performance of the framework is not degraded,and high-level accuracy was reached for the restored traffic.At the last,the possibility was discussed on applying the proposed framework to edge/fog networks.