| Compared with the technology of traditional copy strategy,technology of the erasure code strategy can achieve the same data reliability as technology of the replication in a distributed storage system with a lower cost of storage.As a highly scalable and highly reliable distributed framework,Hadoop adopted a strategy of replication with a copy number of 3 before version 3.0.After version 3.0,HDFS applies erasure code,which makes file blocks in groups of k.The calculation of encoding is performed to generate n blocks which will be stored in n nodes,and the original data can be recovered later by using any k blocks from k nodes.EVENODD adopts a method of encoding based entirely on operation of XOR.The code does not involve the construction of the matrix and the inversion of the matrix,so it has higher efficiency of encoding and decoding,and the code has the property of MDS.Hanxu Hu et al.proposed EVENODD+ that reduces the update complexity of EVENODD,but it is difficult to implement it in a distributed file system because it has many conditions and complicated processes during decoding.In this paper,the method of decoding in EVENODD+ has been graphically explained,so that the process of decoding can be simply grasped by looking at the picture without understanding the original author's formula of decoding,which reduces the difficulty to implement EVENODD+ in a highly scalable distributed system.The framework of Piggyback is based on RS,which improves the performance of repair in RS but has the same redundancy as RS.K.V.Rashmi et al.proposed a Hitchhiker which is based on Piggyback and implemented it in HDFS-RAID.It was found that the bandwidth of download in(n=14,k=10)Hitchhiker during single-node fault can be reduced by 33% compared to the performance of(14,10)RS.This article improves Hitchhiker by further reducing the bandwidth of download during single node failure.The original Hitchhiker is constructed with two RS stripes as a basic unit.This paper takes the idea of vector coding,taking 4 RS stripes as a basic unit,so that the bandwidth of repair in(14,10)Hitchhiker during single node failure can get a further 8% decrease compared to the performance of original(14,10)Hitchhiker.In this paper,the graphical EVENODD+ and the improved Hitchhiker are implemented on the HDFS of Hadoop3,and the comparison among the original two codes,the previous two codes which are modified,the original erasure codes implemented in HDFS in the three aspects of the time of encoding,the time of decoding and the time of reading data for reconstruction is demonstrated with pictures.The result of experiment shows that(5,3)EVENODD+ is about 6% less than(5,3)EVENODD in the time of encoding and the time of updating.The time of reading data for reconstruction of the improved(9,6)Hitchhiker is about 3.83% less than the original(9,6)Hitchhiker in the case of single node failure. |
