Research On Hierarchical Parallel Hybrid Storage System

High Performance Computing has rapidly developed from the PetaFlops age to the ExaFlops age, which has brought serious challenges to the I/O performance, scalability, reliability, and the manageability of the mass storage system. Existing techniques of data storage and data management can not satisfy the requirements in the developed information age. Traditional storage medium and storage architecture has become a major bottleneck of large-scale parallel computer system to obtain high performance. It becomes an urgent task to study the application of new storage medium to overcome the shortcomings of traditional storage medium, the Hybrid Storage technology based on the new storage medium, and the new storage architecture and data management methods to improve the performance of storage system and the efficiency of data management.Based on the in-depth study of a new storage medium, i.e., Phase Change Memory (PCM),this research has designed the Hierarchical Parallel Hybrid Storage System based on PCM according to the I/O requirements in high performance computing. To overcome the shortcomings of current mainstream metadata management algorithms, this research has also designed and implemented the Directory-coding metadata management algorithm, based on the in-depth study of Object-Based Storage.This paper includes the following aspects:(1)This paper has analyzed the difficulties encountered with traditional storage medium under the rapid development of high performance computing, and the trend of diversified development for the storage medium. The PCM with non-volatile, high-performance, high-density and high reliability has selected as the key point of study, based on the research of current various non-volatile memory storage medium. And the technical characteristics and the related work of PCM has been studied.(2)This paper has studied the techniques of metadata management in object-based storage systems. The current mainstream metadata management algorithms, such as Subtree Partitioning algorithm, Hash algorithm, and a number of improved algorithms, have been compared and analyzed. And an idea is proposed to improve these algorithms.(3)This paper has proposed a Hierarchical Parallel Hybrid Storage System storage system by adding a high-speed storage layer based on PCM to the traditional storage hierarchy. We have implemented the strategy for storing objects by using the computational and processing capability of the object storage device (OSD). The OSD can analyze the I/O access pattern, predict the subsequent operations, and pre-fetch the possible objects into the high-speed storage layer based on PCM. We have also proposed the mechanism for adaptive prefetching and object migration in Parallel Hybrid Storage System with the help of I/O pattern prediction algorithm in mining technique based on sequential pattern. The experimental results show the effectiveness of the Hierarchical Parallel Hybrid Storage System architecture and the mechanism of adaptive prefetching.(4)This paper has designed and implemented a metadata management algorithm by Directory-coding, based on the in-depth research and analysis of the current mainstream metadata management algorithms. To compared with the Subtree Partitioning algorithm and Hash algorithm, a simulation experiment has been carried out. The experimental results show its strengths of the operational performance in the directory and the equilibrium distribution of metadata, and the performance and load balance of the metadata management in storage system has been improved.