Research On Network Intrusion Detection Based On Machine Learning

The rapid development of neural network and information technology has promoted the progress of society and brought more convenience to people's life,but at the same time,network attacks also emerge in an endless stream,and network security has gradually become the current research hotspot.Intrusion detection provides an important means of protection and effectively protects network security.Most of the current intrusion detection research based on machine learning methods,there are still some problems in practical application,however,such as abnormal network flow data is difficult to obtain with annotations,real network data attribute feature extraction difficult,existed fitting unsupervised learning training model,the method based on rules cannot detect new attacks,etc.Therefore,in the real network environment,further research on intrusion detection technology based on machine learning has important theoretical and practical significance.Aiming at the problems existing in the current intrusion detection technology in the actual network security application and the actual network security requirements of the Institute of High Energy of Chinese Academy of Sciences,this paper takes the Secure Shell(SSH)logs collected by the real network as the research object,and takes the accurate identification of normal and abnormal users as the research target.In this paper,a novel auto-encoder is proposed to combine one-class Support Vector Machines(OCSVM)with Long Short Term Memory Network(LSTM).AE combined intrusion detection algorithm.Then,an intrusion detection system based on LSTM AE is developed to improve the detection rate.The main work of this paper is as follows:1.Aiming at the difficulty of real network data attribute feature extraction and the overfitting problem in most existing unsupervised models,this paper designs an intrusion detection algorithm based on LSTM AE.First,a feature construction method of statistical value of time period is designed,For example,connection related characteristics based on login behavior can be built by counting the login success and failure times and login duration of an IP in a time window,and content related characteristics based on transmission behavior can be built by counting the variance of transmitted bytes and packet loss,etc,then Pearson correlation coefficient was calculated for feature correlation analysis to complete feature selection,then OCSVM was used to filter some abnormal data to reduce overfitting,and finally OCSVM+LSTM AE model was trained.Compared with OCSVM,AE,LSTM AE and other single and combined models on SSH data set,he detection rate of AE and LSTM after OCSVM filtering training was significantly increased to 83.99%and 87.54%,respectively,while the false positive rate was also decreased to 1.01%and 0.95%.2.In view of the network security requirements of the high energy department of the Chinese Academy of Sciences,which must guarantee the security of a large number of log data at all times,this paper designs an intrusion detection system that meets the requirements of high detection rate and low false alarm rate.On the basis of the architecture design of the security operation and maintenance platform,the system requirements are analyzed in detail and the system design objectives are planned.Firstly,the system architecture and database are designed.Then,technologies and tools such as Bro-cut and MySQL are respectively used to realize data acquisition and storage.One-hot coding and max-min standardization method are used to complete data pretreatment.The intrusion detection algorithm based on OCSVM+LSTM AE proposed in this paper is used to complete the anomaly detection and anomaly management,and finally the intrusion detection system is visualized.After system test and evaluation,the system has been deployed to the actual network environment of the Institute of High Energy,Chinese Academy of Sciences.The actual operation results show that the system can detect known anomalies such as violence,remote login and weak password with the detection rate of about 90%.Through data acquisition,feature construction,algorithm design and other work,this paper finally realizes an intrusion detection system,and the test shows that the system can detect abnormal data in stable operation.