The Performances Of Distributed Big Data Processing Modes In High-speed Traffic Network

With the rapid development of Internet and communication technology,network is closely related to people's lives.The Internet applications produce a large amount of network traffic of user data flow which contains valuable behavior information.The method to process mass data efficiently in high-speed traffic network has become the focus of attention in both academia and industry.As,the research on the performance of distributed computering is deficiency and superficial,it is necessary to do further research by the way of model simulation and data analysis.Firstly,the thesis introduces the features of high-speed traffic network and discusses the challenges of mass data processing in this environment.Then we introduce the related technology of big data processing in brief.Secondly,we analysis the technical scheme of Hadoop and discuss the related factors affecting the performance in detail.We propose a new method to model and simulate Hadoop based on Petri-Net in ordet to predict the performace of Hadoop.Then we prove the simulator is accuracy,efficient and scalable through the contract experiment between simluation and real environment.Finally,we discuss the emerging causes and architecture of Spark which is the burgeoning engine for big data processing.We compare and evaluate the performance of Spark with Hadoop through the actual testing experiment on high-speed traffic network.