Study On Parallel Recovery For Node Failure In Heterogeneous Storage Systems

In the era of Big Data, the mass data with an explosive growth urged to be stored and managed efficiently. The large scale distributed storage technology has aspiration to achieve a great leap. The requirements on the storage system are also more and more stringent, such as storage capacity, I/O performance, data security, data reliability and so son. As to users, in the world where data is everything, data reliability is the most deadly. Given the common presence of failure, distributed storage systems have exploited redundant data to ensure the reliability. In case of failure, it is critical to recover the failed disk quickly and effectively.The main work of the dissertation are the following.1. The design and implementation of Distributed Storage Research Platform. Distributed Storage Research Platform makes it much easy and convenient to develop storage technology. Firstly, it is a special distributed storage system with various encoding schemes, and it provides what a file system can provide. Secondly, it contains an interface of failure recovery to maintain the data reliability and provides varied failure recovery methods. Thirdly, it uses deduplication strategy to ensure a nice space utilization of disk. And at last, it also provides expansion interfaces of disk scaling for the scalability. This dissertation will describe the Distributed Storage Research Platform from all aspects, including its functions and usages, its principles, its design and implementation, which provides a simple and practical experimental environment for the study of Distributed Storage System.2. The parallel and heterogeneous failure recovery for RAID6coded storage systems. With the consideration of both parallelism and heterogeneity, we have designed a three-stage pipelined failure recovery strategy, and proposed a parallel heterogeneous recovery algorithm with the minimum recovery time. Besides, we take a great number of digital simulations and real-world tests to prove the feasibility and effectiveness of this algorithm. This dissertation will describe this algorithm in detail, including the background of the parallel heterogeneous recovery research, the establishment and derivation of the algorithm model, the explicit implementation of the algorithm, the performance analysis and related experiments. Parallel Heterogeneous Recovery algorithm performs much better than any other recovery algorithm in the parallel and heterogeneous environment. It has focused on the parallelism and heterogeneity of systems, for failure recovery optimization, and promoted the system reliability.