Study On The Optimization Of Distributed Storage And Repair Of Failed Nodes

In the era of big data,distributed storage has become the most effective way to solve massive data storage.As distributed storage systems become larger and larger,node failures caused by disk errors are becoming more and more frequent.And then how to repair failure nodes quickly and improve the performances of storage systems has become an urgent problem to be solved.Redundant storage is usually adopted to implement fast repair of failed nodes.Traditional redundant storage schemes,such as copying or erasure codes,each have their own defects and deficiencies.Specifically,replication schemes have high storage overhead,and the erasure codes have high bandwidth transmission.The regenerating codes can achieve the best compromise between storage overhead and bandwidth consumption,but the disk I/O overhead is large during repair,and the repair complexity is also high.Therefore,under the condition that the storage overhead is relatively ideal,this paper further optimizes the bandwidth overhead,repair locality and repair time during the fault recovery of distributed storage systems,the main research contents are as follows:(1)A locally repairable code method based on Fractional Repetition Cyclic Codes(FRCC)is proposed.Specifically,the nodes in distributed storage systems are divided into multiple local groups,and in each local group,FRCC are constructed.Moreover,the failed nodes can be repaired cooperatively by the adjacent nodes in the local repair groups.Performance analysis shows that,locally repairable codes based on FRCC can quickly realize data reconstruction of multiple failed nodes.Compared with the three-copy replication strategy,simple regenerating codes and locally repairable codes based on traditional Fractional Repetition Codes(FRC),this locally repairable codes based on FRCC have advantages in performances of bandwidth overhead and repair locality.Considering that multiple neighboring nodes are failed in the distributed storage systems,the locally repairable codes based on Interleaved Cyclic Convolutional(ICC)codes are constructed.The locally repairable codes can complete the exact repair of multiple adjacent failed nodes,and meantime have multiple repair options for the failed nodes.Moreover,the relatively low repair locality is ensured during repair.(2)Adopting Fractional Repetition Codes(FRC)in heterogeneous distributed storage systems,a class of Heterogeneous Fractional Repetition Codes(HFRC)is constructed.Specifically,the original data blocks are encoded by Maximum Distance Separable(MDS)codes firstly,and the output coded blocks are layered and cross designed to determine their repetition degrees.Moreover,HFRC with repair options are generated based on Fractional Repetition Codes(FRC),including the regeneration layers and the cross-regeneration layers.Theoretical analysis shows that between homogeneous and heterogeneous FRC with the same coding rules can be mutually converted.Compared with Minimum Bandwidth Regenerating(MBR)codes,HFRC can achieve exact repair of the failed nodes quickly,and better performances of disk I/O overhead and repair time during repairing the failed nodes.