Research Of Data Management And Transfer Model Based On Multi-Replica In Grid Environment

Data grid meets the demand for data-intensive tasks with good data sharing and collaboration capabilities, such as high-energy physics, climate modeling and so on. However, because of the dynamic and complex grid environment, node failures and unexpected changes in network occur frequently. So the speed and stability of grid data transfer can't be guaranteed, and it has become the"bottleneck"that restricts grid applications.Replica is the key technology of data grid. It creates local copies of the remote data, reduces network delay and bandwidth consumption, and simultaneously forms a way of multi-replica coexisting gird resources sharing which provides opportunity to resolve transfer problems. So the research of data transfer based on multi-replica becomes an important approach to resolve the problem of speed and stability of data transfer in grid environment.The purpose of this paper was to increase the data transfer speed and stability in grid environment. The Globus Toolkit middleware was used and the research focused on the combination of Replica technology with data transfer. The main works were that:(1) Grid data management and its replica technology were analyzed: this paper summarized gird data management and its replica technology, as well as the involved replica location and selection algorithms;(2) Research of data transfer mechanism in grid: the analysis of the influence to grid data transfer by different resource sharing ways or different transfer protocols was made;(3) Transfer performance analysis of GridFTP protocol was made by experiments: we did experiments about GridFTP parallel transfer and strip transfer. And through the analysis, the importance of this paper's further research was much clearer;(4) A data transfer model based on multi-replica (MRT) and its algorithms were proposed: this paper proposed a data transfer model based on multi-replica, named MRT, and defined the model's elements and mappings between them; Then, the model's multi-level replica location strategy based on locality was designed; Besides these, a heuristic dynamic task allocation algorithm was designed based on heuristic method and probability forecast method. Finally, we made the analysis of complexity of the strategy and algorithm;(5) Design and implementation of MRT model's test system: this paper designed and implemented MRT model's test system from two aspects: the whole and modules. And the experiments testing model's performance were done based on the testing system.Theoretical analysis and experimental results showed that the MRT model had effectively improved the speed and stability of data transfer, especially for bulk data transfer.