Research On Optimization Of Reconstruction For Heterogeneous Erasure-coded Storage Clusters

Since failed nodes' appearance has been a regular event in large scale storage system, storage systems usually use replication or erasure codes to ensure the data reliability. Due to the ability to significantly reduce the cost of storage space for the same availability, erasure codes have been adopted by many distributed storage systems. In addition, with the replacement and upgrading of hardware devices in the storage system, as well as constantly updating of the deployed software, the heterogeneity of actual storage system node is becoming increasingly apparent. Therefore, this dissertation takes the heterogeneous erasure code storage cluster as the research object, and then study on reconstruction of failed nodes and degraded reading.Aimed at reconstruction in the heterogeneous erasure-coded storage cluster, this dissertation proposes a Load-aware Reconstruction Scheme. According to work load and capability of surviving storage nodes, LaRS determines the number of blocks reading from each node, thereby makes the load on each surviving node more balanced, which improves the reconstruction performance. In the reconstruction process, on the one hand, on the basic of considering the heterogeneity of the nodes LaRS once reads data blocks on mutil-strips; on the other hand, according to the response time of each node to determine the load condition of the node,which reflects the node capability. Experimental results show that, compared to existing TRS and FastestRS program, LaRS speedups the reconstruction performance ratios by 1.58 and 1.52.Aimed at degraded reading in the heterogeneous erasure-coded storage cluster, this dissertation proposes an Adaptive Degraded Read Scheme. In the case of intensive user I/O request, ADRS combines all surviving nodes properties into account to unified schedule multiple degraded read request, so that multiple degraded read requests can adaptively match and play all the available resources of current survival nodes, thus reducing the average response time of degraded read operation. Through extensive experimental evaluations in an actual storage cluster implemented three kinds of reading schemes, we validate that ADRS reduces the average response time of 35.39% and 19.05% compared to other schemes.