The Research Of Cloud Based Massive Network Anomaly Traffic Detection Platform

The development of the Internet has given rise to the emergence of cloud computing platform. Cloud services, cloud architecture, cloud storage and a series of solving large-scale data computing platform have been initially formed a system specification. In the network, the data volume of the application service is rising fast, the network security event grows quickier more than once upon a time, and there is also the explosive growth nowadays. As people dependent on the Internet, the harm brought by the economic or national security which is caused by the network intrusion behavior and the attacktion of the network infrastructure is greatly enhanced. How to accurately found the existing abnormal flow behavior in the large-scale network in real time, and effectively solve the problem of security is a very meaningful topic.Based on Hadoop and Spark as the basic framework, this paper puts forward a kind of large-scale network abnormal flow cloud detection platform architecture. From data acquisition, storage, computing to displaying various aspects, we adopt the Hadoop ecosystem of related technical implementation in this paper.In the algorithm designing, this paper puts forward a semi-supervised anomaly traffic detection based on ISODATA algorithm. For known attacks, the algorithm can classfy the abnormal flow type and the normal flow type correctly. For unknown flow, we use the clustering algorithm to dig the abnormal behavior which is separated from the normal network flow. By the simulated experiment and the experimental data in the campus network, we prove the effectiveness of the algorithm.This paper implements the detection environment based on the Spark on YARN cloud modeling. Because the high requirement in the feature of real-time, we designe the cloud detection system model. The Cloud detection platform has been deployed to analyze the Tianjin Educational Metropolitan area network. The platform functions well in detection and precaution the abnormal behavior so as to ensure the normal operation of the network.