Research On Detection Technology Of Network Abnormal Behavior Based On Big Data

The traditional network abnormal behavior detection method can not meet the storage and processing of large-scale data,and because of the single algorithm,the general accuracy rate is low and the false alarm rate is high.To solve these problems,This paper firstly researches the network anomaly analysis platform based on big data,and proposes a network anomaly analysis platform based on big data.Then parallel SA_SVM_RF network abnormal behavior detection model is implemented based on spark.Finally,the validity of the method is verified by the NSL-KDD dataset.The main research results of this paper are as follows:(1)A platform suitable for network abnormal behavior analysis is proposed.Based on the research on the existing big data analysis platform,an analysis platform for network anomaly detection is proposed.The platform includes data collection and preprocessing layer,data analysis layer,data storage layer,and visualization layer.We make full use of big data technology to solve the problem that traditional network abnormal behavior analysis methods cannot effectively deal with in the big data environment.(2)A method of network abnormal behavior detection is proposed.A parallel SA_SVM_RF network abnormal behavior detection model was implemented based on Spark,and the proposed algorithm was compared and tested using the NSL-KDD dataset.Experimental results show that the algorithm can effectively improve the accuracy of network anomaly detection and detection rate,besides reduce the false alarm rate.(3)The network anomaly online detection is implemented.To verify the validity of the model,Jnetpcap and Flume are used to collect data and send it to Kafka,Spark streaming reads the real-time data stream in Kafka and uses the detection model for real-time detection.Realized the parallel collection and real-time detection of massive data.In summary,this paper makes full use of the advantages of big data technology and machine learning in network anomaly detection,effectively improves the processing speed and accuracy rate of network abnormal behavior detection,and reduces the false alarm rate.It improves the detection rate and has certain theoretical significance and practical application value for network abnormal behavior detection under the big data environment.