Research On Virtual Resource Scheduling Of Elastic Cloud Platform

The traditional Internet data center have four weaknesses:low resource utilization,difficult to meet the needs of business development,complicated management and high operation and maintenance costs.To solve these weaknesses,cloud data center come into being which can deliver virtual resources to users via the Internet in the form of services.Cloud computing uses the computer infrastructure hardware and software to form the cloud data center and it has four major characteristics:virtualied resource pool,scalable resource management,pay-as-you-go services and ubiquitous access.Virtualization technology and cloud resource management technology are two key technologies in cloud computing.Correspondingly,it exists two challenges:efficient consolidation of virtual maichines and flexible scaling of virtual resources.In response to the above two challenges,we designed and implemented DartCloud,which is a cross-domain cloud resource leasing platform.We optimized the placement of VMs in the cloud data center,and built ScalaBD based on virtual machine clusters which is an elastic big data computing platform.The contributions can be summarized as follows:(1)We designed and implemented DartCloud to provide users with leasing services for virtual machine cluster.At the resource management level,we implemented a time state based auto-supplement deamon for virtual machine image management and a Web-based SSH proxy server that allowed external networks to penetrate the intranet.At the business level,we proposed a VM state based fine granular virtual resource billing strategy,which includes prepaid mode and pay-as-you-go mode.(2)We proposed a multi-target virtual machine initial deployment optimization algorithm,taking into account two indicators of comprehensive average resource utilization(CARU)and comprehensive resource utilization balance(CRUB)in the cloud data center.Our algorithm can solve the low virtual machine performance,low physical machine resource utilization and high data center energy consumption problems due to the heterogeneous nature of resources in virtual machine brought by multi-tenant application scenarios.In the experiments,we compared our algrithm with CPUPack,MemPack and Stripping three basic deployment algorithms,the results showed that our algorithm can improve the comprehensive average resource utilization of about 5%and get the best comprehensive resource utilization balance of 8%.(3)Big data scalable computing platform ScalaBD can provide storage,batch computing,streaming computing,memory computing and workflow computing services.In terms of storage,we proposed a high privacy and high availability data storage service based on matrix encryption.In terms of batch computing,we analyzed the resource scalability of Hadoop clusters,including scale-out and scale-out methods.In the experiments,we run WordCount,RegexMatch,TeraSort benchmarks and Mahout parallel machine learning applications for scalability analysis,the results showed that in the CPU-intensive applications scale-up method achieve better performance,and I/O-intensive applications have better performance in scale-out method.Finally,based on the virtual machine cluster configuration and big data computing framework parameters,we propose the concept of double-layer parameter adjuster to further improve the resource utilization of the cluster and reduce the leasing cost of the cluster.