Research Of Job Scheduling Technology In Hadoop Platform

With the rapid increase of data size and the great wealth of the applicationtypes,the demand of information to consumer and corporate far exceeds the ability ofthe original platform. enterprise-level or individual-level is overwhelmed by more andmore applications and platfom.the Hadoop platform as an open source frameworkwhich handle large database running on the cluster becomes more and morepopular.however,Hadoop is still young,There are many places have the possibility tomake changes and improvements.Through extensive research,this paper introduces the definition of cloud copmutingand key technologies,In-depth researchs on the Mapreduce and the HDFS of Hadoopcore..then,this paper analyze the design ideas and the advantages and disadvantages ofexisting three scheduling algorithms:FIFO,Fair-Scheduler,Capacity Scheduler.For the problem that Hadoop parameters too much to set up and schedulingalgorithm too much difficult to choose,propose a sampling algorithm based on hugeamount of data.and design the improved Hadoop framework which join the strategicchoice layer to solve the two problem.finally, it repackage and experiment the Hadoopplatform.Compared with the old version,The results show that there is obviousimprovements in dealing with massive data.