Reseach On Mapreduce Parallel Computing Platform For Cloud Computing

Cloud Computing(CC) is the center of development for the next generationInternet service standard. Based on existing Internet technologies, CC usesvirtualization technology for building Virtual Machines(VMs) on top of hardwareinfrastructures. These VMs are flexible and scalable in the sense that at any instant,they can be configured to provide the exact amount of required computing andstorage resources. Thereby, these resources can be rented out to customers exactlyaccording to their computation and storage requirements at real time. The21stcentury is an information explosion period, and that CC has demonstrated that it issuitable for running Parallel Computing to tackle the large data processing problem.However, traditional Parallel Computing(PC) framework like MapReduce(MR) isdesigned for static and homogeneous environment in which nodes are physicalmachines. However, CC provides elastic and cost-effective VMs as computingresource. In CC, VMs’ resource can be dynamicly modified by VM Monitor tosatisfy actual request, which means CC is dynamic and heterogeneous. Hence,existing MR mechanism should be modified to perform Parallel Computing on thecloud.In this thesis, we propose an elastic computing mechanism for MR in whichVM’s computing resource will be dynamicly modified hotly using XenAPI. Twosteps are taken to achieve elastic compute scheme. In the first step, MR job’s typewill be classified as CPU-intensive or Data-intensive type, which means differenttype tasks will ask for different resources and VM will be equipped with appropriateresource configuration before job is running. In the second step, VM’s resource willbe modified according to workload when MR jobs are running. Givencost-efficiency requirements, many experiments will be conduct to acquire resourceworkload limits and some other factors. Further, resource cannot be fully utilized inMR because of MR’s static Slot mechanism. A dynamic Slot mechanism is proposedin this thesis to achieve full utilization of resource.