Increasing data reliability and recovery I/O performance in erasure coded storage systems

Ensuring data reliability has always been a key concern in large storage systems. There have been numerous schemes put forward to make data more durable, which include replication and/or erasure coding. Remote auditing can be used to ensure that data is available and free from corruption and/or deletion. We outline how to make remote data auditing more robust against small targeted deletions. But recently more and more attention is being paid to the recovery problem, i.e. given that a (hardware or software) failure has occurred, the cost of recovering the data affected by that failure should be minimized. While there are lots of facets to such an open ended problem, we focus on the I/O aspect of it. We examine ways of minimizing the amount of I/O needed for recovery in the context of an erasure coded storage system. This involves analyzing the recovery characteristics of existing codes, as well as a new class of erasure codes which have been specifically designed for optimal recovery I/O performance. We provide an in depth experimental evaluation of the theoretical results to show that reading a minimal set of symbols during recovery does indeed translate into practical savings in reconstruction I/O cost.