The Research On Storage Management Of Virtualization

With the development of computer hardware (CPU, RAM, disk array, high-speed network and so on) and software technology (virtualization, parallel computing and distributed system), emerging computing patterns such as grid computing, service computing and cloud computing developed rapidly. Virtualization can provide service consolidation, dynamic allocation of resource, enhanced security isolation to those new computing patterns to resolve problem such as security, isolation and resource allocation.At the same time, with the explosive growth of global information, the requirement for data storage space of enterprises and individuals is increasing more and more, and the payout for storage device and its proportion in IT cost are both growing larger. Now the rapidly growing payout for storage device and high energy consumption of device, and on the other hand the low utilization of storage space poses new demands and challenges for the storage management technology. So a lot of new storage management technologies and service patterns have arisen, such as thin provisioning, free space reclaimation, data de-duplication and cloud storage.As other cloud computing, cloud storage integrate many heterogeneous and cheap servers and storage devices to construct a lot of virtual OS instances for enterprises and individuals to provide continuously application and storage service. It can not only satisfy the storage requirement of many middle & small scale enterprises and individuals but also cut down the payout and energy consumption of storage device. But then the stress of expenditure and energy consumption will be transfered to servers of cloud storage. Thin provisioning can purchase less storage capacity up front, defer storage capacity upgrades in line with actual business usage, so it's a method for optimizing utilization of available storage. It's an important trend to apply thin provisioning for cloud storage platform to optimize utilization of storage device.Although thin provisioning can optimize utilization of storage device, there will also be amont of free space because of frequent file delete operations. Currently only a frew thin provisioning support space reclaimation, and most of them reclaim free space only when volume is migrated to another server, so those reclaimation technology can't reclaim free space effectively.To increase storage utilization farther, in this thesis proposed a Virtual Storage Resource Flow on-Demand Model (VSFoD) to strengthen thin provisioning. VSFoD integrate all storage resource of a server into a unified storage pool, and from which construct logical volume as the virtual block device (VBD) for every virtual instanc (virtual machine, VM). Based on this model the VBD can extend on demand, and when the storage is in short supply, some VBDs of VMs can be shrinked and even the storage of other nodes in the same LAN can be shared to satisfy the extend and creation requirement of other VMs. That is to say the storage resource can "flow" from a VM to another, so VSFoD can optimize utilization of storage device more effectively than thin provisioning, and then cut down the expenditure and energy consumption of storage device more effectively, and are more suitable for cloud storage.Xen is a high performance and open-source server virtualization project, many commercial products and scientific research projects are all based on Xen, including typical Amazon EC2 (Elastic Compute Cloud) and S3 (Simple Storage Service) in cloud computing. The prototype of VSFoD described in this thesis is also realized based on Xen.First of all the prototype improved the current logical volume manager(LVM), and when creating or extending a logical volume it can consider the different performance of every physical storage device to improve read/write performance, the evaluation result warrant the conclusion. On the other hand when creating or expending logical volume user/administrator can customize the expected mapped physical storage area, and then optimize the LVM.Then the prototype improved the virtual block device management of Xen and make VM can modify its VBD capacity after the VBD is extended or shrinked, otherwise the VM need to reboot to know the updated capacity of its VBD, and this will interfer users' access. In addition, the prototype also give a new performance isolation method among VMs which is more sutiable for Xen, evaluation results demonstrated that this method can guarantee the disk IO bandwidth be proportional shared by all VMs according their weights more effectively than other IO schedulers in Xen.After the VBD is resized, the upper file system must be also resized to match the updated capacity of the VBD, at the same time the file system resize should try to reduce the interference to user's read/write, and then VM can utilize the dynamic updated VBD. This thesis will describe a method to shrink the most common file system for Linux - ext2/3 online, and realize the shrink online into Xen. Evaluation results demonstrated that the method can correctly shrink ext2/3 online, and won't suspend or block user's read/write process, and in the worst it only bring down the IO performance only 5%~10%.