Research And Implementation On Data Placement And Load Balance Strategy For Multicloud Storage System

High-availability multi-cloud storage systems are the current research hotspots in the field of big data storage.However,with the continuous development of big data application models,new challenges are presented to the existing multi-cloud storage mechanism: on the one hand,different big data applications have large differences in data read and write operations,making their I/O mode different.In order to ensure the access performance of applications,it needs to design flexible data placement schemes for the multi-cloud storage system according to the I/O mode of different applications;on the other hand,with the continuous increase of application data,the problem of data unavailability is unavoidable.It needs a reasonable load balancing strategy to reduce the risk of performance bottlenecks caused by data placement.Therefore,it is necessary to place the data of different applications in a multi-cloud storage system reasonably and achieve effective load balancing,which has important value for big data storage.For this reason,this paper studies the data placement and load balancing strategies in the mixed I/O mode of a multi-cloud storage system.The main contributions of this paper include:Firstly,In order to solve the problem of data placement in multi-cloud storage under mixed I / O mode,a fine-grained I/O aware data placement strategy for multi-cloud storage system is proposed.The strategy includes two stages: fine-grained I/O awareness algorithm based on decision tree and adaptive data placement strategy in mixed I/O mode.The former uses decision tree model to accurately perceive the I/O mode of the data,and the latter can design a reasonable data placement strategy.Experimental results show that the strategy can accurately divide the data I / O mode according to the characteristics of the data,with an accuracy rate of 96.57%,and reduce the data read and write access delay by 20% to 50% compared with the solution without any optimizations,which optimizes the overall performance of the storage system.Secondly,In order to solve the problem of multi-cloud storage load balancing in mixed I/O mode,a multi-cloud load balancing strategy is proposed.The strategy separately designs a periodic load balancing strategy and a temporary load balancing strategy.The former can solve the problem of uneven data distribution,and the latter can effectively deal with a large number of the sudden I/O requests of big data applications.Experimental results show that the strategy effectively eliminates the performance bottleneck caused by data placement and improves data access efficiency by nearly 20%.Thirdly,Based on the above two theoretical results,this paper designs and implements a multi-cloud storage data placement optimization system.First,build a multi-cloud environment including Ali OSS,Baidu BOS,Huawei OBS and other cloud storage providers.Second,implement and deploy the theoretical results of this paper in the system to provide effective multi-cloud storage services for big data applications.Finally,through the experimental tests in the real environment,the effectiveness of the theoretical research work in this paper is verified.In short,this paper has conducted in-depth exploration of data placement and load balancing mechanisms in multi-cloud storage systems,and designed and implemented a multicloud storage data placement optimization system.Relevant experiments show that the mechanism proposed in this paper effectively improves the performance of big data access,and can handle a large number of sudden I/O requests,which can provide an effective method for data placement optimization and load balancing of existing multi-cloud storage systems.This paper can make useful contributions to the development of big data storage management.