Font Size: a A A

Research On Optimization Of Big Data Storage Replica Strategy In Cloud Environment

Posted on:2019-09-14Degree:MasterType:Thesis
Country:ChinaCandidate:S X LiangFull Text:PDF
GTID:2428330566496017Subject:Computer application technology
Abstract/Summary:PDF Full Text Request
The advent of the Big Data age brings opportunities and challenges to mankind.Cloud storage provides an ideal storage solution for Big Data.Availability and performance are important considerations for users using cloud storage.The replica technology in cloud storage can not only maintain the high availability of the system,but also improve the performance of the system as a whole.Compared with static replica technology,dynamic replica technology can meet the requirements of data access in the complex environment of cloud storage.The dynamic adjustment strategy of replica factor and the problem of replica placement are the focus of the research of replica technology,which is also the main research content of this thesis.Based on the analysis of the problem of dynamic adjustment of the replica factor and the lack of the static replica mechanism of the existing Hadoop distributed file system,this thesis forecasts the access heat of the file with the principle of time locality,and adopts different adjusting strategies for different heat files,which is accomplished by screening and adjusting two stages.It improves access performance while avoiding waste of storage resources.Experimental results show that the improved replica factor adjustment strategy can reduce the average response time of the system,and can improve the performance of data access effectively.In this thesis,aiming at the problem of replica placement,the limitations of existing replica placement strategies in heterogeneous environments are analyzed.Combined with statistical knowledge,the heterogeneous properties of nodes in a cluster are quantitatively evaluated.On the premise of following the given basic principle of replica placement,replica is placed on the basis of the different evaluation values of the comprehensive performance of the nodes reasonably.The experimental results show that the improved replica placement strategy can make the distribution of replicas more reasonable and balanced under the precondition of ensuring the overall availability of the system,and it is helpful to improve the localization ratio of map task and enhance the execution efficiency of MapReduce.
Keywords/Search Tags:Cloud Storage, Hadoop, Dynamic Adjustment of the Replica Factor, Replica Placement
PDF Full Text Request
Related items