Distributed File System Zd-dfs Design And Implementation

With the development of Internet, the storage and retrieves of magnanimous data from Internet has become the primary load of many network services. The backend file server will easily be the performance bottleneck of whole system when data flow and the read/write request from Internet grow quickly. But the traditional file systems are difficult to meet the performance requirement of magnanimous data. At the same time, current general distributed file systems don't supply a good support to the magnanimous small files from the Web environment. So, we need to develop a specific file system which can support the magnanimous small files well. This paper introduces the design and implementation of ZD-DFS cluster which adopts the structure of one meta data server managing the storing nodes. ZD-DFS shields the detail of resource locating, load balancing, distributed storage, distributed transaction management, data migration and data consistency. It provides uniform programming interface to user. This system implements the functions which satisfy all the requirements of the web environment by the cooperation of meta data server and distributed store nodes. The meta-data server manages the global information, resource locating, data migration and load balancing. It also manages all the backend storing nodes. The meta data server provides global support to distributed storage, and distributed transaction. With the management of the meta data server, ZD-DFS offers the file system interface the upload, read, update and delete functions. Distributed transaction processing of ZD-DFS supports two phase commit protocol and XA specification. Fault tolerance mechanism is used to process the failure and exception. ZD-DFS assure the data consistence by using the lock mechanism, version countermark, and heartbeat protocol. ZD-DFS cluster system achieves high performance for storage and retrieves of magnanimous data. It is well scalable. The storage and computing capability can be easily extended. This system designed in this paper will be a good backend system for large network applications.