Design And Implementation Of Client On Aliyun Distributed File System

With the demand for computing and storage growing bigger and bigger at the time of big data and cloud computing, the distributed file system has been the solution to storage for system with plenty of data. Aliyun Company has been developing its own distributed file system that provides storage service for its own cloud products and big data processing for years. I participated in re-design and re-implementation of client on Aliyun distributed file system. And this paper is based on this refactoring.This paper discusses the background of distributed file system. Then introduce and compare several pre-existing distributed file systems. This paper explains Aliyun distributed file system, its advantage, and the client part of this system. And introduce the related technologies used in this system:random access read/write and stream read/write, protobuf, Vectored I/O, remote procedure call and chosen of programming language. This paper analyzes the requirements for client on Aliyun distributed file system, describes functional requirements and non-functional requirements, and does the use case analysis. For design of client, there is module diagram, sequence diagram and so on to describe the whole design. And some important modules are described detailed. For implementation of client, this paper describes the implementation of random access file part of RAFFile module.There are four file types in Aliyun distributed file system:temporary file, log file, normal file and random access file. The operations of open, close, read and write are the main functional requirements of client. The operations of random access file are the most complicated and important in client. And the support of random access file makes Aliyun distributed file system different from others, as there is almost no pre-existing distributed file system supporting random access file. This paper describes the implementation of random access file in client, following the internal function invocation sequence of three operations:open, read and write.