Research On Key Technologies Of On-demand Deployment Of Virtual Machines In Distributed Environments

As an important virtualization technology, system virtual machine (VM) has many good characteristics, such as cross-platform, strong isolation, legacy application support and mobility, etc, and has bee increasingly applied to many distributed computing fields, such as grid computing, cloud computing, cluster computing and mobile computing, etc to solve those problems difficult to be solved by traditional approaches, such as system migration, security and isolation, system management, etc.To successfully apply the VM technology to distributed computing environments, the VMs must be able to be deployed fast to the target computing nodes and to run with low performance penalty on them. VM deployment is a process of distributing the VM images which include the VM state to the target computing nodes and then starting/resuming the VMs on them. Since a VM image usually has a large size, traditional VM technologies and deployment mechanisms will cause high deploying cost in distributed environments, and are difficult to achieve fast VM deployment and efficient VM running at user sides, especially in large-scale computing environments or low-bandwidth and high-latency network environments.This thesis proposes an on-demand VM deployment mechanism based on fine-grained splitting of VM image. The mechanism deploys part of the VM image rather than the whole so as to achieve fast and efficient VM deployment in distribute environments. The work of this thesis mainly involves four aspects: the VMM (Virtual Machine Monitor) support for on-demand VM deployment, the VM image and software management mechanism, the on-demand VM image distributing mechanism, and the mechanism for efficient VM running at user sides. Specifically, the thesis makes the following contributions:Firstly, the thesis designs a new COW virtual block device (VBD) for the block device virtualization layer in VMM to solve the problem of disk access performance degrading linearly with the depth of COW disk hierarchy in traditional COW VBDs. The new VBD optimizes the block addressing approach and the block storing granularity in the traditional COW VBDs, for which it can split the traditional monolithic large-sized VM image into many small-sized COW disk images without impairing the access performance of virtual disk. To better support on-demand VM deployment in distributed environments, the new COW VBD is also extended to support demand-fetch and shared memory management functions. Theoretical analysis and experiments show that the new COW VBD has better performance than the traditional COW VBDs, especially in the case of deep COW disk hierarchy.Secondly, long-term or frequent software management in VM environments may result in large numbers of COW disk images, causing disk space wastage and making the management of VM images difficult. Based on the generic COW disk model, the thesis proposes two COW disk reclamation mechanisms to reclaim the reclaimable disk space in the old-version COW disks and to maintain the number of COW disks in a specific range. Experiments show that the two mechanisms can effectively reclaim the old-version COW disks and control the number of COW disks so as to greatly reduce the disk space wastage and to ease the management of VM images, supporting long-term or frequent COW VBD-based software management in VM environments.Thirdly, the thesis proposes an on-demand VM deployment mechanism based on fine-grained splitting of VM image to support fast and efficient VM deployment in distributed environments. The mechanism utilizes the new COW VBD to split the traditional monolithic large-sized VM image into multiple small-sized COW disk images at the basic granularity of a single kind of software, and then distributes them to the user sides on demand so as to achieve fast VM image distributing. Since most disk access requests can be satisfied locally and demand-fetch technique is only used for a few missing blocks, the VM can run efficiently at user sides. An effective working set-based optimization approach is also proposed to reduce the deploying cost of those kinds of large-scale software. Experiments show that the on-demand VM deployment mechanism can greatly reduce the VM deployment cost and can better satisfy the VM deployment requirements for fast VM image distributing over the network and efficient VM running at user sides, supporting fast, on-demand VM deployment in distributed environments.Finally, to reduce the waiting time and also to achieve better VM performance at user sides, the thesis proposes a demand-driven VM image prefetch mechanism to further optimize the on-demand VM deployment process. The prefetch mechanism reduces the waiting time at user sides via demand-fetch technique, and improves the VM performance by prefetching the COW disks of those kinds of software being used at user sides. The thesis proposes an access frequency and dynamic priority-based prefetch target recognition algorithm to improve the accuracy of recognizing the prefetch target, and a dynamic adjustment mechanism to adjust the prefetch amount dynamically during the prefetching process so as to reduce the effect on the prefetching process caused by factors such as network condition, etc. The thesis also proposes a P2P-based prefetch mechanism based on an extended BitTorrent protocol to speed up the prefetching process. Experiments show that the prefetch mechanism, especially the P2P-based prefetch mechanism, can greatly reduce the waiting time and can also obviously improve the VM performance at user sides.