Design And Implementation Volume-Based Hierarchical Storage System

With the rapid development of information technology,data grows explosively, how to store massive data efficiently takes great challenge to the design,construction and running for large-scale storage system.In large-scale storage systems, the various computing,transmission and storage devices have very big difference in physical characteristics; At the same time, the access specifications of the storage devices for the actual applications are not uniform, they vary in time and space, it's not wise to store all the data in high performance devices. The present of hierarchical storage can solve the above problems properly ,it can efficiently monitor the data access specifications, according to which it will optimize the configuration of storage resources with heterogeneous characteristics.By analyzing the shortcomings of the file-level data migration mechanisms, we designed a volume-based hierarchical storage management system, it can automatically assign the storage resources according to the data access specifications of the logic volumes, thus enhance the overall efficiency of the storage system.We first introduced the applicable environment of hierarchical storage system, discussed the key technologies for implementing the system, analyzed the issues related to realizing the access specifications and storage device performance based hierarchical storage management system.Secondly, we designed and implemented the system prototype,the system is composed of three parts, the client,the manager and the storage resource agent, each part has a few modules, and each module achieves specific functionalities, and the user requirements monitor module,storage virtualization module and data migration module are the core. The user requirements monitor module collects the users'access specifications to achieve the hierarchical storage based on user access frequency; The storage virtualization module classifies devices according to their performance to achieve the unity of the heterogeneous resources; The data migration module collects user access frequency and migrate data between the devices of different performance. The system can support online migration and migration velocity control, thus effectively ensure the users'access performance.Finally, we checked part of the systems' functionalities, evaluated the system's performance, and analyzed how data migration affected the users' access performance. At the same time, the experiments also showed that the foreground application throughput could improve 10% -40% by controlling the data migration rate.