Research On Data Distribution Strategy Based On Storage Virtualization

As human society enters the information era, the amount of data generated each year are growing with a surprising speed. Therefore, it is necessary to make an extensive data management for the massive data, which can self-adapt to the dynamic change of the large scale storage system. Storage virtualization is an effective management, which can map the whole storage device as a virtual storage pool. However, as massive data are generated by different applications in the virtual storage devices, how to effectively store the data according to its feature and how to help users to quickly localize data are becoming a challenging problem. For this problem, this paper studys data distribution strategy based on storage virtualization. The main contributions are as follows:At first, the paper proposes a Data Distribution Strategy Based on Storage Virtualization (DDSBSV) model and makes a detailed analysis of the storage virtualization and binding mechanism of storage resource for the model. The basic idea of DDSBSV is as follows:Firstly, the design of a kernel loadable module for storage virtualization; Secondly, the reasonable allocation of storage load to different virtualization disk binding sets according to characteristics of I/O load; Thirdly, the load balance of the storage media in the virtual disk binding set; Finally, the reasonable data migration among the virtual disk binding set according to the sequent/random accessing frequency and the data access frequency statistics. Therefore, the model provides three stages of data distribution strategy to meet the needs of heterogeneous applications and make full use of all storage resources.Next then, the paper implements the detailed design for DDSBSV in the kernel of Linux2.6. Via the idea of virtual memory management and the expansion of Device Mapper mechanism, the paper realizes storage virtualization on the basis of which the paper provides the idea of binding storage resources. By analyzing the characteristics of I/O load, the weight of virtual disk binding sets can be dynamically changed. Combining with the round-robin scheduling algorithm that based on the weight, the paper realizes the fairness of the distribution of data and hierarchical data storage. By adopting the dynamical interval mapping algorithm, the paper realizes the load balance in the storage media of different virtual disk binding set. By adopting weighted directed graph (WDG) and the Depth-first searching algorithm, DDSBSV can obtain the sequent/random accessing frequency of I/O. On the basis of the degree the paper realizes the data migration among disk binding sets thereby further optimizing the data distribution and realizing hierarchical data storage.Finally, the experiment makes a contrast between DDSBSV storage system and Linux LVM2. The results of the experiment demonstrate that the throughput, bandwidth and average responding time of DDSBSV have been improved remarkably comparing to that of LVM2. Meanwhile, DDSBSV can guarantee the storage QoS needs, quick accession and self-adaption in the large scale networking storage system.