The Design And Implementation Of Xunlei Offline-download Distributed File System

With the widespread use of computer network in social living, distributed application becomes a mainstream gradually. Because of its inherent shortcomings,such as single fault point and low Scalability, traditional centralized server system more and more limits the development of network service to multi-aspect. In order to meeting the need of distributed server system,which has high performance, high extensibility, high reliability and high usability, people also turn to use distributed server system slowly.A distributed file system named Xunlei Offline-download Distributed File System (XLFS) was developed based on Linux kernel. XLFS is a scalable distributed file system for large distributed data-intensive applications. It provides fault tolerance while running on inexpensive commodity hardware, and it delivers high aggregate performance to a large number of clients.This thesis firstly designed the whole framework of distributed file system, and used a design sheme different from old ones for file system aiming at the specific requirements of Xunlei offline-download project. First, the scheme of single master server and several chunk servers has satisfied scalability for file system while simplified the designation. Second, in allusion to the characteristic of offline-download data that much appending and little amending, the file system specially provides atomic appending operation. Third,servers interact with each others in the form of lease, which could keep the consistency of data at the level of chunk, not byte. At last, different from most of old file systems, XLFS provides fault tolerance for data in engineering by verify code.The research achievements are mainly in the following aspects: First of all, mastered relevant technologies of distributed file system, and simulately achieved the mechanisms such as atomic appending, rubbish callback and lease. Then designed and completed a distributed file system, including the master server, chunk servers and client interface. Last, aiming at the problem of application procedure call, achieved the FsShell intetrface of xunlei offline-download distributed file system similarly to linux operation system API. the FsShell interface was tested in this file system, and the system ran well in the testing environment.