Research And Improvement Of MapReduce Scheduling Mechanism On Cloud Computing

Since cloud computing since2006has been proposed to change the network concept iswidely used in all aspects of the Internet. Various IT industry leaders in the deployment andcloud computing technology, cloud computing technology in core services. Cloud computingis a new generation of Internet technology, the research focus. Therefore has importantpractical significance for the study of the cloud computing technology.One of the key technologies of cloud computing MapReduce parallel data programmingmodel proposed by Google. Hadoop platform is an open source imitate Google’s MapReduce,is also the most widely used in the world, an open source cloud computing platform.Therefore of great significance to the research and optimization for the Hadoop platformMapReduce model.This paper first introduces cloud computing concepts and background knowledge, andthen presents and analyzes the key technologies of the Hadoop platform.MapReduce model in the study of the Hadoop platform mechanism after inadequacies,proposed an improved scheme, named Dynamic adaptive scheduling algorithm (AdaptiveCapacityAlgorithm Based on Priority, hereinafter referred to as ACBP). This algorithmrunning in accordance with the actual operation of dynamically changing the execution setnumber of jobs, the adaptive system task scheduling mechanism, and this article on the theHadoop default speculated mechanism has been studied to improve against the inadequaciesof the speculated mechanism makes the discriminant backward task whichever side, to avoidunnecessary system resources consumption. Start the backup task, the allocation of a node, itis necessary to consider the node system load, considering the remaining computing power torun backup tasks rational allocation of nodes, avoid starting an invalid backup tasks, thuseffectively improving system integrationFinally, experimental verification algorithm performance of the algorithm under theexpected conditions, and a first-in, first-out scheduling algorithm and fair schedulingalgorithms and capacity scheduling algorithm derived experimental results contrast. Theexperimental results show that the algorithm of this paper as more good performance thanFIFO scheduling algorithm, but compared to the fair scheduling algorithms and capacityscheduling algorithm compared, or there are some limitations. However, a good performancein a heterogeneous environment. To achieve the intended purpose of the experiment.