Font Size: a A A

A Dynamic Data Migration Strategy For Large Data In Cloud Computing Environment

Posted on:2017-03-09Degree:MasterType:Thesis
Country:ChinaCandidate:J F ZhangFull Text:PDF
GTID:2278330488950087Subject:Computer software and theory
Abstract/Summary:PDF Full Text Request
There are many application types in big data,two applications are special.One of it is that the needed data should be stored in some specific data center because of confidentiality or other reasons; Another application has cohesion between data set, it means that, the output data sets of a task is the input data sets of another task. And in the execution of this two category of applications, if we migration the needed data of the task before execution, it can reduce the task completion and make the system more balance.Cloud computing with a mass storage space and high-performance capabilities, has become the natural application platform of big data. Cloud computing core idea is: we have to spend much more to transfer data between different data centers than the task scheduling. Therefore, these two kinds of application in the use of cloud computing platform has met a lot of problems. Main show is in the cloud computing core idea. To perform these two kinds of special applications, it is inevitable to implement a data migration.In face of multiple data centers, such applications meet some challenges in data migration. It mainly manifest in below aspects:(1) In the migration process, it needed to visit network many times, so the number of network access must increase;(2) In the process of migration, the amount of data needed to migration is huge, compared with the amount the bandwidth of the network transmission between the different data centers is certain, so the frequent cross data center transmission is bound to bring time consumption;(3) In the data migration, we need migrate some data in a data center to other data centers, and the load of data centers are change,.It is likely to bring load imbalance problem of data center.According to these challenges, this article first has carried on the description, analysis and modeling to problem. Then put forward a data migration strategy according to the different big data application, solve the global time consumption of data migration, the number of network access and global load balancing these three parameters and use genetic algorithm to optimize data migration scheme.Finally, do the cloud computing simulation experiment under the Cloudsim experiment platform. The result shows that the proposed strategy makes the task completion time reduced by 10% and the the data transmission time accounts for the proportion of the total time is reduced.When the amount of data sets is increase,the proportion can reduces to 50% or less.Network access number lower than Zipf and reached stable, in global load, the variance of the node’s store space closed to zero.
Keywords/Search Tags:cloud computing, Big data, Load balancing, Data migration, Network access, The data set
PDF Full Text Request
Related items