Research On Durable Data Storage Mechanism In Cloud Storage System

The virtualization services provided by cloud storage, greatly simplify the application process, and reduce construction costs, which brought great convenience to users. But frequently security incidents make people worry about the reliability of cloud storage. Such as hardware failure, malicious attack, personal operational errors, power outages, fires and other natural disasters, leading to failure of one or more storage nodes, make the possibility of the collapse of the whole system greatly increased. Therefore, the cloud storage system should have a strong data storage protection mechanism to ensure system reliability and data availability.Durable data storage mechanism can effectively ensure the system reliability and data availability in cloud storage system. The durable data storage mechanism's related technologies include data fragmentation, data distribution, correctness verification, and maintenance of redundant data. The availability of stored data is an important indicator to measure system performance, and data's availability mainly rely on the nodes' organization around the system and the treatment of redundancy data. The durable data storage mechanism implement a technology for splitting and merging file by erasure codes, and the distribution based on directory for redundant data, and an efficient method for correctness verification. The mechanism can decode to recovery the lost data by existing data. By setting the adaptive size of buffer, sub-file reading method of encoding data can improve the coding speed obviously. Meanwhile, the content of the original file is distributed to ensure privacy. For the improvement of the core erasure codes algorithm, the mechanism obviously increases the average encoding rate during the splitting and merging of files.Theoretical analysis shows that the data redundancy scheme based on erasure codes to ensure data availability has the obvious advantage. Compare to the replicated scheme, erasure codes can reduce the overhead of storage space and network bandwidth for the same data availability. System performance evaluation's results show that erasure codes can reduce the overhead of storage space and network bandwidth more than 50% for the same data availability, comparing to conventional backup mechanism based on replication. Setting buffer in the process of coding can improve the coding efficiency significantly, and the optimized algorithm's average coding rate in the file fragmentation and merging are increased by 22% and 19%.