Research On Optimizations Of Data Reconstruction For Erasure-code Storage Clusters

With the increasing scale of erasure code storage systems, the large-scale of nodes leading to the node failure and the data loss become the normal. Traditional data reconstruction methods lead to failure long recovery time and higher front-end I/O latency. Therefore, in order to ensure the high availability and the online response performance of the Erasure code storage system, designing new efficient reconstruction optimization schemes to improve system recovery performance and front-end I/O access performance has the important significance to the scientific research.For the above problem, in storage cluster based erasure code for the study, in order to optimize the reconstruction performance and user access performance as the goal, we proposes two optimized schemes. And we call them offline reconstruction based on pipeline I/O optimization scheme and online reconstruction based on memory redirection optimization scheme.In the offline reconstruction based on pipeline I/O optimization scheme(PSUH),the reconstruction process is decomposed into a set of linear combination operations and each of the computing operation is assigned to each surviving node, finally all surviving nodes complete the reconstruction process by pipeline I/O. Using the pipeline reconstruction chain, it can not only avoid the replacement node' acceptance bandwidth becoming the bottleneck of reconstruction, but also can improve the response performance of the survival node disk. Experimental results show that compared with the traditional offline reconstruction mode, PUSH can accelerate the speed of reconstruction performance at least 92%.In the online reconstruction based on memory redirection optimization scheme(RAM-RS), using the I/O separation strategy, redirecting the failure user requests to the memory area which is consisted of a number of pre-allocated memory from all survivals and separating the reconstruction process from user request process can reduce the interference of user access process to reconstruction process. Using the reconstructed user access data which is cached in the memory area to response subsequent user access requests can speed up the response to the user. Experimental results show that comparedwith traditional online reconstruction mode, RAM-RS can accelerate the speed of reconstruction performance at least 20% and reduce the average response of 78%.