The Studies And Realizes Of The Large-scale Distributed File System

The appearance of the Internet in the 70's of 20 centuries changes people life style greatly, push forward the human society into an information Age. Along with development of the network and computer information, the network storage is becoming the development point in the recent years. The data center of the Fujian telecommunication hopes to through an united of data saving terrace provide business and enterprise with the network storage service. Thus it is necessary to design the large-scale distributed file system. A lot of research works are carried out for the distributed file system, an put forward many techniques and method. After studying the latest research completely, the conclusion is made that the current distributed file system can't satisfy with large-scale network storage application in the aspect of load balance and scalability. After investigating the current distributed file system, we put forward the large-scale distribute file system( LDFS) that adopts distributed layed management model instead of traditional central resource management model. The global address is adopted to meet with public network storage service; and the striping of file is adopted to guarantee the performance of reading and writing when the storage nodes join; metadata and data are stored separately to get better use of the storage device and the network, improve the performance of the system, lower the cost of the system. For improving the system performance better, particularly accounting for the demand for the public network in load balance, failover and scalability, we think that the work in that is concentrate in the storage nodes, but the research in the metadata server is limited. In the case of proliferation of data, metadata server will become the bottleneck to influence the performance. For meet with the large-scale network storage in metadata service , we put forward using a metadata server cluster consist of several metadata servers to provide coherent metadata service, and manage the metadata server cluster with the management scheme of hashing of the logic partition. The scheme divides storage space of metadata into several logic partitions, and partitions are allocated to metadata servers. The logic partitions and the metadata server are assigned with the weight according to their load balance. When the load balance is changed, through the transform of function and parameter of weight, the control of logic partition is shift from the metadata server of heavy load to that of low load. but the process does not involve the movement of metadata. In addition, the mechanism is offered to satisfy the metadata server cluster with the scalability, reliability and so on.