Research On Data Layout Strategies For Cloud Storage System

Cloud storage is a storage service provided by cloud computing platform, which integrate a large number of different types of storage devices in the network to provide data storage and business access functions through the cluster, grid technology and distributed file systems. Currently, cloud storage has become a hot research topic in cloud computing applications.The high-speed, security, high reliability, high availability and low power consumption are the key performance indicators currently pursued by cloud storage system which are closely related to the data layout technology in the cloud storage system. Although there are a number of researches and discussiones in this regard, but the existing data layout strategies still have some problems need to be further studied and resolved. They did not taking into account the needs of users as well as system performance and energy aspects of cloud storage systems. Therefore, optimizing the data layout strategy for the cloud storage environment to build high-performance, high reliability and low energy consumption cloud storage system is an important and urgent task.This dissertation mainly studies the data layout techniques for efficient storage in cloud storage system, focus on four aspects that are static file data layout, dynamic file data layout, replica data layout, and the application of data layout policies in the data storage system with low power consumption. The aim is to optimize the performance of the cloud storage system and reduce the average response time of the system, and makes the cloud storage system having high-speed, high availability and reliability, and low power consumption characteristics. The main research work and innovations include the following four aspects:(1) Propose a static file layout strategy, called SFLS (Static File Layout Strategy), which aims at accelerating file access speed in cloud storage system and taking load balancing of the system into consideration. SFLS strategies can not only enhance the access rate of the cloud storage system, but also balance the load between nodes under different workload conditions. The basic idea of SFLS is as follows: SFLS first divides disks into two groups which used to store hot files and large files, respectively; Then divides the files into two types of hot files and large files accordding to the files’ service time and frequency; At last, it assignes the files sorted by the service time in Round-robin or Greedy way to the two groups of disks. This approach avoids the phenomenon of "Hunger" caused by hot files waiting for large files and also maximizes the utilization of the disk. It can decide which disk group should be accessed based on the size or heat of the file, which speeds up the access of the file and improves the access performance of the storage system. In order to evaluate the performance of the proposed file layout strategies, cloud data storage and management modules are added on the basis of the existing CloudSim toolkit to build a cloud-based data platform for simulating data cloud. The expanded CloudSim effectively facilitate users to evaluate their striping strategy, data layout strategy and replica management strategy in the task scheduling of the available cloud data center resources.(2) To better optimize the data layout design of the cloud storage system, a dynamic file assignment strategy, called prediction-based dynamic file assignment (PDFA) is proposed, which aims at minimizing the mean response time of the cloud storage system with the consideration of load balancing between data nodes under different workload conditions. First, an analysis model of file assignment and file access to cloud storage system is proposed, which can be used to evaluate the performance metric in terms of mean response time of data accesses for different file assignment policies. Then, a load prediction model for cloud storage system which can estimate the future loads of data nodes is present. Third, a dynamic file assignment strategy based on the overall load estimation which can choose an appropriate file layout scheme under different workload conditions is described. At last, the performance of the newly proposed dynamic file assignment strategy together with the famous HP and SOR is evaluated. The analytical and experimental results show that the proposed PDFA performs best in terms of mean response time.(3) A multi-objective optimized replication management strategy referred as MORM based on the artificial immune algorithm is designed to improve the data availability and reliability for Cloud Storage Cluster. Several factors such as mean file unavailability, mean service time, load variance, energy consumption and mean access latency are took into account to capture the relationship among replica number and replica layout with these five performances. Five objectives are built up for optimization which provides the advantage to search for solutions that yield close to optimal values for these objectives. MORM can assign replicas to different storage nodes, which obtains a high degree of redundancy and fault tolerance.(4) A low energy consumption cloud storage model named LECCSM was proposed. This model not only dynamically switches the state of data nodes through load prediction to enable energy saving at the single data node level, but also dynamically reconfigures the cluster according to the current workloads and the costs of the whole system so as to achieve energy efficiency at the data center level. It also proposes the optimal replica management strategy for replica layout. Then, the proposed three layout policies are applied to the LECCSM and the re-distribution strategy was analyzed for these data layout strategies to adapt to the energy-efficient cloud storage system, which makes the proposed layout strategies not only adapt to conventional storage systems but also adapt to today’s low energy consumption storage systems.