Repair-efficient Codes Construction And Performance Evaluation Based On Distributed Storage Systems

Distributed storage system is a kind of system that the storage devices are interconnected through network.However,the devices are often unreliable,which lead to frequent failure of storage nodes.In this case,data recovery operation proceed in distribution storage systems regularly.Traditional distributed storage systems maintain data consistency and availability by keeping file copies.Even though replication is simple and effective,it may cause large scale of data redundancy and increase repair cost.Based on the information above,how to repair the disabled nodes in distributed storage systems effectively has become an urgent problem,which has very important research value and practical significance.Coding technology,which has been proposed and become an effective way to solve the data repairing problem.In the field of coding technology,researches on fault-tolerant technique of regenerating codes have become popular in recent years.At present,studies on regeneration code have been divided into two parts: the minimum storage regenerating codes(MSR)and minimum bandwidth regenerating codes(MBR).These two aspects represent how to reduce data redundancy and how to reduce bandwidth during data recovery.In terms of overhead between storage and network resources,efficient use of network resources can lead to more financy savings.Therefore,we focus on MBR codes and select four kinds of repair-efficient regenerating codes in this paper for theoretical and practical performance study.The repair process of these coding schemes is to obtain a copy of lost data accurately or just through simple computation,so they have better performance and lower bandwidth consumption.At the same time,they provide better storage capacity and more flexibility on choosing storage mode than other minimum bandwidth regenerating codes.In this paper,we study the performance of the repair-efficient regenerating codes based on the distributed storage system.The main work is focused on the following aspects:1.According to the properties of included coding schemes,we summarize their similarity and plan a unified coding framework in order to realize the file-level operation.By abstracting the nodes and network connection in the distributed storage system through undirected graph theory,which means regarding vertex as storage node and edges as the interconnection relationship of nodes,the topology structure of the distributed storage system is obtained.At the same time,the process of file operation is discribed using matrix operations.The performance of coding scheme during encoding,decoding and repair single-node failure is theoretically analyzed and summarized in the end.2.In this paper,a distributed storage system based on repair-efficient regenerating codes is designed and implemented with modular design.We introduce design process and algorithm of function implementation.The system is deployed on a server cluster which storage devices are interconnected through local area network and uses the opensource object storage project OpenStack Swift to simulate network storage environment.After deployment,we evaluate file-level operation performances of each coding scheme with appropriate parameters according to the unified coding framework.Finally,we propose analysis of results and the best storage policy in data repairing.We hope our research can provide referance for the follow-up study in the field of regenerating codes.