Research On The Flexible Virtual Resource Management Mechanism For Complex Scientific Computing Appllication

Since the 21st century,the scientific computing has become the major method of scientific data analysis and processing.It has become increasingly important in promoting the development of human science research.When running on datacenter,scientific computing applications need to consume a lot of computing resources due to its massive computation requirements.Furthermore,the computing resource consumption has risen sharply with the increasing scale of scientific computing applications.However,current data centers are usually limited in computing resource capacity.As a result,how to achieve efficient resource management and fully utilize limited resource to meet the growing resource consumption of scientific applications become an urgent problem.With the development of virtualization technology,resource virtualization can virtualize physical machines into virtual machines to realize the flexibility resource configuration and rapid resource deployment.In recent years,utilizing flexible management of virtual resources to achieve the efficient use of physical resources has become the major method to achieve efficient resource management.At present,the research on virtual resource management has made some progress,but it is still a kind of "coarse grain" management method,which is easy to cause waste of resources from the following three aspects.First,most of the existing works lack the accurate analysis on resource requirement of complex scientific computing application;as results,they cannot achieve on-demand resource provision based on application's specific resource requirement,which may cause resources over-provisioning and lead to the waste of resource.Second,most of the existing works lack the efficient optimization for online large-scale heterogeneous virtual resource allocation,which easily leads to poor resource allocation and low utilization of physical machines' resource.Third,most of the existing works lack the flexible resource adjustment according to the dynamic changes of application's resource requirement,which results in inefficient use of resources.Therefore,this dissertation focuses on how to utilize of the advantages of virtualization technology to achieve "fine-grained"resource management and reduce waste of resources.To address the above problems,this dissertation studied the resource provision optimization,resource allocation optimization and resource use optimization of the resource management problem for complex scientific application.We designed a flexible virtual resource management mechanism to achieve finer granularity resource management and improve datacenter resource utilization.Specifically,this thesis research works are carried out in the following four aspects.Firstly,in order to improve the resource provision efficiency,we combine the resource requirement analysis and the virtual resource provision to design an on-demand resource provision mechanism for complex scientific applications,and achieve the resource provision optimization.Secondly,we design an online large-scale heterogeneous virtual resource allocation mechanism to deploy virtual machine upon resource request's arrival,which can achieve fast resource allocation to realize resource allocation optimization.Thirdly,we design a hybrid resource pool based virtual resource adjustment mechanism,which can carry out resource adjustment according to dynamic change of actual resource usage to improve resource usage's efficiency and achieve the resource use optimization.At last,we design and implement a prototype of flexible virtual resource management system to evaluate the effectiveness of the theoretical approaches proposed in the dissertation.Overall,this dissertation presents a flexible virtual resource management mechanism for complex scientific application and achieve efficient resource management.The proposed strategies and algorithms of the resource management mechanism are also evaluated through a series of simulations and testbed experiments.The evaluation results demonstrate that the proposed strategies and algorithms can efficiently improve the resource management's performance when running complex scientific applications on datacenter.Meanwhile,our studies provide a strong guarantee for the efficient execution of different scientific data processing applications on SEU datacenter and provide an effective solution for resource management in large-scale datacenter.