Research On Non-structure Data Replication Method Of Multi-Datacenter

Internet is changed by Web2.0. More and more non-structure data are being generated by interaction among users. These non-structure data are stored in datacenters all around the world, in which there are a large amount of servers. Data replication between datacenters from different locations is becoming more and more a pressing need for data backup and performance requirement.We improve methods how HBase filters and stores data when replicating. An effective way for data replicating is discussed extended to inter domain, which is then improved to be with multi-cores and shared-cores scalable. Finally, we bring forward a dynamic priority queue method based on the probability of priority increase.The main work of the paper includes:Filter and store column family as a unit when replicating based on this column store feather. Propose an addressing plan based on a two-dimensional hashing algorithm, which makes store and address relatively centralized with higher reading rate and less concurrent connections. A method how we choose a set of nodes from target datacenter randomly is explained.Propose an effective replication method that replicates data directly through mesh links in replication network. Study how inter domain network performs with multi-cores or shared-cores. Then an algorithm to build source-core replication tree of shared-core inter domain network is built up. Finally evaluate its scalability according to the routing entropy.Build up a priority increasing probability based dynamic priority queue for replication task according to the theory of EDF. Methods how the probabilities are generated and corrected are studied here. The test shows that it is in line with expectations. Then relationships between the number of combined replication tasks and their increasing probabilities are discussed.