The Design And Implementation Of A Network-Based Distribute Storage System

With the Internet exploding in size and reaching into every aspect of the society, digital data stored are growing at an unprecedented rate. Many organizations are under continuous pressure to expand their storage systems as geometric demands for their services grow and their data sets swell relentlessly. The small capacity memory devices with inferior access efficiency are unqualified on the requirements. In recent years the computer industry has made significant advances in magnetic recording technology, and how to store the massive resources has become an issue requiring urgent solution. The experiments show that in many organizations most of the local disk space on client workstations is only used for operating systems, application programs and temporary files, which in total take up fifty percentages of disk space, and a large number of storage spaces are wasted.Motivated by the increasing pervasive resource wasting, to allow the application not to purchase expensive dedicated facilities to lower storage costs significantly, we propose a relatively small scale distributed file storage system with a central server. In this thesis we determine system architecture based on the actual scenarios. On performance issues, the system metadata and chunks are stored separately which get better use of the storage device and the network .The system distributes the access to files to multiple machines which guarantee the performance of reading and writing. On the problem of load imbalance for a distributed system, we adopt chunks heat transfer and replication strategy file based on system load level through the transform of function and parameter of weight and the number of times to access files which balances the load on the nodes effectively. To improve the system reliability and utilization of storage space, the system adopts built-in data partition and redundant replica strategy to achieve reliability and high availability. Copying the chunks dynamically is utilized to ensure that the data have a higher availability. In the ease of use, the system provides file operations interface to the application and the client uses a graphical interface.In this thesis, we describe the design goal and function modules of the system, and the entire implementation process is described in detail. We uses ordinary computers in the laboratory to test the system from reliability, performance, scalability and capacity, proving that the system has a good bandwidth of accessing to files and expansibility, achieving the desired design goals by solving the problem of massive data storage effectively.