The Design And Application Of A Scalable Distributed File System

With the rapid development of mobile Internet, the burgeoning Internet data showed its growth in geometric scale, meaning the advent of Big Data era. Consequently, single file storage server has been unable to cope with the increasing size of data, the distributed file storage system was built into use, which has become one of the basic components of Internet applications nowadays. Due to differences between Internet application services, the same distributed file system’s efficiency varies from business applications. To ensure the efficiency of the file system, a specific distributed file storage system is specially designed for a specific application service.This paper has designed and implemented a scalable distributed file system, after an in-depth study in the reliability, availability and scalability of distributed file systems. While considering the specific scenario that the size of most files in current Internet application resource does not exceed 100 KB, the file system designed in this paper was improved to ensure its performance in deal with frequently store, read and write of massive small files.The main work of this paper include the following.1. Method of the smooth expansion of file system. In Internet applications, the scale of business will be enlarged with the increasing of time, file storage system will also require expansion. Great scalability is an important indicator of the performance of distributed file system. With hardware directly added, the file system designed in this paper is capable of smooth expansion according to specific application needs, no other configuration needed.2. Method of the read and write process of massive small-size files. In Internet applications, most of the files are small files no more than 100 KKB in size. In this scenario, frequently read and write of file data is needed. To ensure efficiency of massive small file to read and write, this paper has been optimized.