Algorithm Research Of Network Wide Traffic Anomaly Monitoring Based On Hadoop

With the continuous expansion of network scale, the level of network securityprotection requirements more and more highly.The traditional intrusion detection arebased on single link or node, but in large scale network most anomalycharacterization in single link or node is not obvious, which led to the traditionalintrusion detection can not take into account the large-scale network, so it isnecessary to cloud computing is introduced into intrusion detection.Efficient, highfault tolerance, scalability and high reliability as well as the characteristics of Hadoopopen source cloud computing platform on the massive data processing has had theability to help improve the handling massive amounts of data, so the Hadoop cloudcomputing platform is imperative to introduce intrusion detection.Firstly,this paper mainly studies the two key technologies of Hadoop: HDFSstorage framework and the MapReduce computing framework.Analysis anddismantling the main step of the Multiscale Principal Component Analysis(MSPCA),and based on the MapReduce implementation of the parallel algorithm ofMSPCA.Secondly, do the experiment of the scalability and speedup test on theparallel algorithm of MSPCA.Finally, in the prototype system verify the parallelMSPCA algorithm for the detection of abnormal flow.