Research And Implementation Of Erasure Code Based On Non-uniform Protection Strategy In Cloud Storage

With the rapid growth of data today,there is no doubt that data means value,people focus far more on saving data safely and reliably.Compared with the traditional storage system,cloud storage system has high performance,high scalability,high availability,large capacity and so on,so it has been widely used.Cloud storage systems typically use three copies of backup technology and erasure code technology to ensure the reliability and availability of their data.However,the erasure code technology in the current cloud storage system follows the traditional coding method of communication field,and does not design for the characteristics of data's non-uniform failure.Just from this point of view,the author has carried on the thorough research in this paper,the main work is summarized as below:1.This paper improves LRC(Local Reconstruction Codes)and designs an erasure code based on nonuniform protection strategy—UFP-LRC(Unequal Failure Protection Based Local Reconstruction Codes).Without adding extra storage space,UFP-LRC divides the raw data blocks into logical groupings of different sizes and provides different levels of erasure protection for different groupings,thereby giving more protection to data on the storage nodes which have high failure rate,while making them less costly to repair/recover.2.With the Hadoop cluster as the overall architecture,this paper designs a cloud storage disaster-tolerant system based on UFP-LRC erasure code.The data files in this system are classified into cold data files and hot data files according to the user's access frequency.Then,on this basis,the cold and hot data judgment technology is proposed,which can accurately judge the cold and hot state of the data file by scanning the file table in the WebServer server.Cloud storage disaster-tolerant system uses UFP-LRC erasure code technology and three copy backup technology to store hot and cold data files to improve their reliability and usability.3.The function test of cloud storage disaster-tolerant system is carried out,and the three performances average recovery cost,recovery failure rate,availability of UFP-LRC code,LRC code and RS are tested and comparatively analyzed.The test results have reached the expected target.Finally,the author summarizes the full text and looks forward to the next work.