Research On Replica Management In Distributed File System

With the vigorous development of new applications based on Internet and the Internet of Things, global data has immerged explosive trends, and big data has become the prominent feature of application systems. Distributed file system is the core technology of unstructured data storage, and it supports massive file storage through distributed architecture. Meanwhile, as one of the key technologies of the distributed file system, replica management can enhance reliability and fault tolerance, andit also plays an important role in load balance and service performance.This paper analyzes the function and strategy of replica management in distributed file system, considering lots of factors like user access, load balancing and system cost, and then proposes a new replica management method.Firstly, a dynamic replication strategy based on heat of files is presented, and it includes heat description, heat updating and replication strategy. A multi-frequency heat algorithm is proposed to quantitatively describes the heat of files by considering file access frequency of different cycles. Meanwhile, files with different heat, use updating policy with different frequencies. Based on the heat of files, the dynamic replication strategy includes adjustment of replica number, multi-frequency enforcement mechanisms, replica placement strategy and replica consistency. The experiments show that the multi-frequency heat algorithm effectively resolves the contradiction between survey cycle and heat accuracy. And explosive access problem is also well resolved. It makes the replication strategy more reasonable.Secondly, a real-time replica strategy taking load balance into consideration is provided. Specifically, a dynamic application-sensitive load balancing algorithm and a real-time replica strategy are included. The dynamic application-sensitive load balancing algorithm automatically adjusts current application by dynamically updating the weight of hardware configuration. And the real-time replica strategy is based on the load balance algorithm, including the cache mechanism for high-heat files and a real-time replica scheduling mechanism. The experiments verify the effectiveness of the load balancing algorithm and the real-time replica strategy, and prove that the proposed algorithm and strategy has a great promotion to the system performanceThe replica management method developed by the paper is implemented in JTang distributed file system. The throughput, response time, load balancing, and other aspects of the system have certainly improved. JTang cloud disk, which is an application based on JTang file system, has been put into use by more than100enterprises and has a good effect.