On The Disk Failure Recovery Of RDP-based Distributed Storage Systems

Building and maintaining a reliable, extensible, low-cost and high-performance distributed storage system is the main objective of background application of cloud computing, big data and Internet companies. Distributed storage system has distinct advantages over the centralized storage system. However, the unreliable components of distributed storage system lead to many exceptional situations in the running process of the system. Among the exceptional conditions, disk failure is the most frequent one. Therefore, recovering disk failures efficiently is the major issue in large distributed storage system.Distributed storage systems are facing frequent single disk failures, making the recovery of single disk failure become a hot research point. One of the most important factors which has to be considered in an erasure coded storage system is the data blocks read from the system are optimized. So the research points of the paper are the following aspects:First, the paper studied the solutions about the data recovery of single disk failure in erasure coded storage system. By considering the reading performance of storage devices in distributed storage system, the paper design a new recovery scheme which is based on disk sequential-read. Our proposed recovery scheme is able to minimize the amount of data being read, and also preserve the disk access sequentiality during failure recovery. And we focus on the RDP code and design the sequentail-read based recovery scheme. To validate the practicality of our proposed recovery algorithm, we conduct various experiments in DSRP, a real erasure coded distributed storage system. We further show that our recovery algorithm improves the recovery efficiency, compared to the conventional hybrid recovery with the random disk accesses.Second, we design and develop the DSRP system, a distributed storage research platform. DSRP can not only operate files and directories like other normal file system, but also support the advance functions, such as encoding and decoding, data recovery, system deduplication and scaling. Meanwhile DSRP provide us an experiment platform of a variety of algorithms in data recovery, deduplication and scaling, with detailed data statistics.