Research On Load Balancing Technology In Distributed File System

In the time that I/O sub-system, especially the storage sub-system has gradually become the center of research on computer system architecture, with the rapid development of disk access technology and network connecting technology, high-performance, scalable, high-availability, easy-management and networked storage-cored computer system architecture has become the main trend of network server system step by step. Distributed file system that is one important part of storage system provides high-performance, scalable and high-availability file service for applications. Therefore, the thesis aims to study how to improve the scalability of storage system through load balancing to obtain the high effectivity of system resource, including designing the scalable file system architecture and studying the load balancing strategy.In view of the ratio of system resource used by metadata operations versus file data operations in NFS service, and metadata consumes approximately 50 percent of cpu time, a new type of networked storage system architecure that deals with metadata and file data separately is provided, which is named BW(Blue Whale) networked storage system. BW system is based on networked storage device and it owns many advantages such as high speed data fetch, data and management share, scalability of service and storage space and management, dynamic ability and so on comparing to typical direct attached storage (DAS) since data is centrally managed on the network. Moreover, the article describes the prototype implementation-BW1K networked storage system and the file access mechanism in it.For large storage system, there are many metadata servers (MS) to manage metadata operations, so metadata management should be distributed evenly in MS collections. While the load of metadata server results from the access to metadata which is mapped on the metadata server, the paper firstly analyses the characters of file access load in the network application background, then points out shortcomings of static mapping strategy of metadata to MS system, which is used in Slice and Lustre storage system, thus proposes a kind of dynamic metadata mapping strategy. The strategy is completed by Binding Server (BS) and MS collections, and it takes into locality of file access as well as the load status of MS. Thurther, only the active metadata is mapped, which alleviates the memory burden of BS and also materializes the flexibility of load distribution. In the consequent test, the dynamic strategy performs excellent load balancing competence under metadata-intensive load and SpecWeb99, at the same time BS doesn't become the bottleneck of system.The thesis also discusses some research about load dynamic migration and describes some design considerations, and thus the above flexibility of load mapping allows the feasibility of the second step for load balancing----load migration.