Research On Logical And Physical Evolution Of Evolving Storage System

The physical and logical organization structure of current storage system is static, and static organization structure cannot depict a continuous-changing system. Besides, current storage system lacks corresponding mechanisms satisfying the changing storage requirements. To solve these existing problems, ESS (Evolving Storage System) mainly study and research on issues of how to construct and organize a dynamic, adapting to changing application storage system, what data distribution optimization policies are applied, and how storage system replace and add new components, etc. ESS system aims to provide user a storage system with high capacity, high performance, high availability and adaptation to various workloads.According to the concept and key idea of ESS, and based on OSSI Model (IEEE Open Storage System Interconnection Reference Model), the system architecture of ESS is build. The function of each component of ESS system is described in detail, which gives much convenience to the system software implementation. The evolution policies could be partitioned into physical evolution and logical evolution. To design and implement ESS storage system, three main issues should be addressed. (1) What architecture is applied by ESS to manage the storage disks in mid/large scale, and to serve to the workload in a large range. (2) How ESS system adjust the data distribution to adapt to the dynamic change of system workload. (3) How to replace the failed disks in ESS system with heterogeneous disks, and how to add new disks to system, thus to achieve ESS system physical evolution.RAID takes advantage of multiple disks parallelism to improve the I/O performance of storage system, and applies data redundancy policy to assure data availability. To adapt to the rapid development of application that require high storage capacity and performance, ESS system still has to apply RAID technology. At the same time, ESS system uses the concept of distributed virtual disk in cluster file system for reference, and proposes an adjustable system architecture constrcted by arrays. ESS constructs all system disks into several SubRAIDs, which constitute a unique global storage space of ESS. The configuration of SubRAID is totally the same with traditional RAID, but each SubRAID can adopt appropriate data placement scheme and performance optimization policy according to the number, capacity, performance of its disks, and the characteristics of the workload it serves. Thus, through SubRAID of different configuration serving different application, the whole ESS exhibits optimized I/O performance to all of its clients.To provide applications with satisfying service quality, the organization and distribution of storage system need to change with the dynamic change of output and input of external data stream. ESS applies"two-level"data migration policy. The fluctuation of workload is responded by the data migration inside SubRAID; larger range of workload toss is then adapted through data migration among SubRAIDs. The internal SubRAID data migration policy is based on PMSH algorithm, which utilizes the characteristic that outer zone of disk have higher data transfer rate. PMSH migrates the stripe units with high access rate to the zones of stripe-located disk with high data transfer rate dynamically based on the"hit"of stripe unit. According to user request size and read/ write ratio, the policy of migration among SubRAIDs migrates data among SubRAID with different disk number, different storage capacity, performance, data placement scheme, and performance optimization policy. Through these two-level data migration, the organization and distribution of data with different attribute can change accordingly with the change of external input and output data stream, thus implement the logistic evolution of ESS.A main contradiction that ESS system aimed to overcome is the contradiction of the rapid development of system component and the relative fixed system architecture. The storage system of current architecture would not enhance its system performance with the update of components. About component, ESS system mainly considers disks. The rapid development of disk technology and the increasing requirement of storage capacity and performance cooperate, and result in three type of dynamic changes in the disks in storage system: the addition of homogeneous disks, the addition of heterogeneous disks and the replacement of heterogeneous disks. The scalability and heterogeneity of system disks of ESS make it adaptive to disk dynamic changes. applies three algorithms (DAA, HDAA, e-HDAA) respectively to add (homogeneous or heterogeneous) disks or replace heterogeneous disks; Further more, through using the added or replaced disks fully, ESS system optimizes the storage capacity and performance without importing too much overhead to the running storage system; thus to achieve the system physical evolution.