Research And Implementation On Technologies Of Metadata Management In Distributed File System

By connecting many computers together, distributed file system can provide storage service with a uniform interface, large capacity, high performance, high-availability and high scalability, which meet the need of large scale application. Distributed file system has been a difficult and popular study point in storage fields. To address the different access features of file data and metadata, recent distributed file systems are divided into to parts: the data storage system and the metadata management system. To handle increasing metadata requests and keep the coherence and reliability of metadata operation, the design and implementation of metadata management system is very important. This article designed and implemented the metadata management system of distributed file system LandFile, and studied the key problems and solutions of metadata management system in distributed file system. This article mainly focuses on the following aspects:The energy-saving-based load balancing strategy of metadata server cluster: in large scale application, energy cost has been a more and more important problem. Based on the dynamic metadata management, we proposal an energy-saving load balancing strategy. By merging the workload of two metadata servers into one and turning off the other metadata server when the overall load is low, the overall energy consumed is saved. The sleeping metadata server will be added into the cluster when the overall load increases. The carefully designed dynamic metadata management ensures a smooth transition when the server is added and removed. Experiment shows that the energy cost is dramatically reduced in energy-saving mode while the performance is not affected.The coherence of metadata management: Metadata may have multiple replicas in the system. A few kinds of metadata operation may involve two metadata servers. We designed the primary-node based cache structure and implemented the update strategy of multiple replicas. We also implemented a two-phase-commit protocol to ensure the coherence of distributed metadata operation.High reliability metadata management: We studied the nodes management, failure detection and failure recovery method of metadata server. By using the nodes management strategy based on region autonomy and log-based failure recovery, the failure node can be replaced and recovered in a short time, the system service is still available during the failure.Using the load balancing strategy, coherence strategy and reliable management strategy, we designed and implemented the metadata management system, which includes metadata operation server module and journal module; we also refined the load balancing module.The metadata management system researched is an important component of 863 topic:"the Development of the New Generation Collaborative Supporting Environment of Business Operation, Management,and Control".