Research On Data Transmission Optimization For Collaboration Of Heterogeneous Cloud Storage Services

In recent years,with the rise and rapid development of cloud computing,various types of cloud storage services have appeared one after another.And the heterogeneity of these cloud storage services not only makes users face problems of selection diffi-culties and vendor lock-in,but also limits different cloud service providers to connect and share various resources to provide pervasive computing.To cope with these issues,various collaboration modes of cloud storage service based on interface standardization or middleware have been proposed,which can improve the service quality,reliability and scalability of various cloud platforms.However,these collaborative modes are mainly in the conceptual design stage.In practice,the clouds ' public interfaces generally have poor access performance and high instability.The phenomenon brings great challenges to cross-personal cloud file collaboration,hybrid public cloud-based file hosting,and data transmission scheduling for multiple cloud storage nodes and other real-time collaborative scenarios.In view of the above opportunities and challenges,this dissertation focuses on how to make heterogeneous cloud storage services work together efficiently to optimize data transmission,including the following innovations:(1)Aimed at the need for file collaboration among users of different personal clouds,real-time collaboration of frequently edited files is achieved by overcoming the low ef-ficiency of the clouds ' public access interface API.First of all,according to the key observations from real-world measurements of inefficient APIs,a proximity-aware cloud proxy deployment scheme and an inter-proxy advanced transfer protocol are proposed.On this basis,control mechanisms of redundant update elimination and file conflict han-dling are also designed.A system is implemented based on these solutions to support file collaboration among a wide variety of popular personal clouds,with the performance of end-to-end cross-cloud file synchronization approaching or even exceeding the intra-cloud file collaboration.(2)Aimed at the users desires for efficient file operations with a low storage price,cost-efficient file hosting is achieved by combining two different types of cloud storage services and carefully tweaking their usages.A hybrid cloud storage approach based on file size is first designed,and advanced transfer schemes including relay-based download and caching,as well as optimization of file transfer operations are proposed for its large-file download performance bottleneck.Control mechanisms of redundant operation request elimination and filesystem consistency maintenance are also designed.A system is implemented based on these solutions to provide cost-efficient file hosting services on top of Amazon S3 and EFS,with very low cost close to S3 and good filesystem operation performance that is close to or even better than EFS.(3)Aimed at the problem that the access latency of typical cloud storage service nodes is highly unstable,a set of algorithms optimizes data transmission for storage nodes of multiple clouds.Among them,two transmission optimization algorithms are first proposed to solve the problem of minimizing data transmission time among multi-node cloud,and between users and distributed cloud nodes.A scalable online dataflow scheduling algorithm is further designed to balance the timeliness of transmission and system overhead.The scheduling system based on these algorithms has obviously better performance than the comparison algorithms,while low overhead guarantees its good scalability.