Research On Key Technologies Of Distributed File System Oriented To Cloud Storage

Cloud computing is a hot topic in recent research and applications. Cloud storage is the lower layer of cloud computing system which supports the service of the other layers above it. Meanwhile, it is an effective way to store and manage heavy data. So it focused even more attentions from some researchers.Cloud storage is a distributed file system with complicated architecture. Firstly, it is implemented on top of the cloud computing infrastructure which is based on cheap, virtualized and unreliable physical hardware. Secondly, it should supports huge server scale, efficient heavy data storage. All of these challenge the key technologies of the system architecture and modules design.A detail research is firstly given on the technology of architecture of distributed file system oriented to cloud storage. This paper proposes a generic architecture of cloud storage based on the distributed file system of master-slave mode by using multiple proxies of metadata server. Theories and analysis show that the scheme has high availability and reliability.To address the different access features of metadata and file data, recent distributed file systems can be divided into two parts:the metadata service layer and the chunk data storage service layer.For the former, the paper studies the metadata replication technologies based on the proposed architecture. Paxos, a consistency algorithm used in many cloud storage sytems, is worth researching. This paper improves the basic paxos by using the supervision mechanism of service nodes and optimizing of communication agreements. Thus, messages quantities of the proposal of metadata and the waste of network bandwidth are greatly reduced. Analysis and simulation results show that it solves the metadata consistency problem efficiently.For the latter, the paper researches the key technologies of efficiency storage. The disadvantages of normal storage scheme of fixed chunk size is showed first, then the paper proposes a new dynamic storage algorithm with variable chunk size.Analysis and simulation results show its high usability that it has can improve the parallel computing performance, balance uneven load of the chunk servers caused by hot files, and generates little file pieces.