The Research Of Dynamic Resource Management Key Technologies In Cloud Computing

Resource Management scheme in Cloud Computing environment is a method to plan, distribute and regulate system resource according to the demand of users and the various parameters of system. It is a core technology, which has great influence on system performance. However, there are some problems to be solved in this area especially based on Hadoop architecture.The first problem is the storage resource lacking of flexibility. Because user’s demands for different data are also different, so that some of the data files could become "hot spots". It is unreasonable to threat all the data files as the same. Cloud Computing needs dynamic storage scheduling scheme, which differentiates demands for different data. The second, metadata is a key issue in Cloud Storage. Most of the current metadata management mechanism is master-slave architecture with only one central node. And some Multi-NameNode metadata management mechanisms have inadequacies in metadata division, persistence and fault recovery. Third, as the specification of resource provided by Cloud Computing Service Provider is different from user’s demand, there will be some fragment resource in system. The system resource utility would be improved if we could take some scheme to organize and allocate those fragment resource. This paper has three innovation points according to the problems discussed above.For the "hot spots" problem caused by the number of replica could not satisfy the soaring request from users, we present a Cloud Computing storage resource replica dynamic scheduling scheme based on HDFS. There are two sub algorithms which is the number of replica dynamic adjustment algorithm and replica placement algorithm. This scheme could improve system stability and file access throughput.The experiment result shows this optimized scheme lower the access delay obviously than the original scheme with some cost of storage resource and network workload.For the problem of metadata management, this paper introduced a distributed metadata management scheme. This scheme distributed metadata into several NameNode according to their ability, and provides an efficient inquiry algorithm for metadata to improve system availability and reliability. More than that, this scheme takes a replica scheme based on group and a load-balance strategy to ensure system fault tolerance and scalability. The experiment result shows the multi-NameNodes system with this metadata management scheme could improve the proformance than the original system facing the huge metadata operation requests.For the problem of fragment resource, this paper brought a distributed scheduling scheme with QoS constrain based on optimized Chord under the resource coupling strength and the demand of fragment resource organization. The simulation result shows that this scheme could improve the system resource utility obviously.