Research On Replica Optimization Strategy In Data-intensive Computing

Rapid development of Internet and popularization of broadband speeds up the process of networking and informatization in all walks of life. The expansion of network data scale poses significant challenges to the computer at the same time. The ability to manage huge amounts of data has become a performance bottleneck in the development of computing power, and data processing systems has developed into data-intensive computing systems. In such a context, Data-intensive Computing (DIC) emerges and causes wide concern.Data management is a core issue in data-intensive computing systems. Replica management technique is widely used in data management and is effective. Replica management includes replica creation, replica selection, replica replacement and consistency maintenance, it can improve the data reliability, balance network load, reduce remote access latency and bandwidth consumption significantly.In full knowledge of replica management technology in Data-intensive Computing environment, in this thesis we will discuss the replica selection and replacement issue. Considering the lacks of existing policies, we propose new optimization strategies. The main work includes two aspects:(1) Proposes an improved replica selection strategy based on ant algorithm for Data-intensive Computing environment. Considering the infinite positive feedback feature of ant algorithm, we choose a replica according to a probability formula. The new method can avoid too multifarious access to a replica, which will lead to network congestion eventually and affect other data transfer tasks. We extend the mainstream grid emulator OptorSim and implement the proposed algorithm, and compare our new strategy with other replica selection algorithms by simulation experiments.(2) Proposes a new algorithm named LRULR based on LRU (Least Recently Used) replica replacement strategy. New strategy considers the distribution of replicas in data grid as an essential factor in replacement. New method can increase the hit rate and access efficiency of replicas; it can also reduce file copies and bandwidth consumption. It’s main idea is to replace the replicas, which are in the least recently used list and has the least copies in the entire grid, when the memory capacity is insufficient. This thesis implements the LRULR strategy in OptorSim and compares it with LRU.This paper presents optimization strategies for the replica selection and replacement respectively, and implements them in simulation platform. The comprehensive experiments in OptorSim show that the proposed algorithms have certain advantages in reducing the mean job time, reducing network bandwidth consumption and balancing network load.