Research And Implementation On A Hybrid Cloud Storage Middleware System Based On Information Decentralization

Cloud Storage,as an important application of cloud computing model,recently has been widely used by enterprises and individuals.Cloud Storage has provided enterprises and individuals with unprecedented convenience.Cloud Storage utilizes server cluster technology,grid technology,distributed file system,Internet to connect different storage devices and it is a great data storage solution to enterprises and individuals.Compared with the traditional storage solution,cloud storage has strong scalability,cost-effective,high accessibility,etc.However,when enterprises or individuals want the Cloud Storage service providers to totally manage their data,they must confront severe problems such as data security,the reliability and availability of service providers and so on.What's more,users have to concern that whether their data will be locked by the service providers.In this thesis,we try to provide a hybrid cloud storage scheme,which is based on Information Dispersal Algorithm(IDA)to address the issue of security cloud storage.The scheme not only can satisfy the industry demand for high capacity data storage,but also guarantee the reliability and security of data storage and using.The thesis firstly introduced the popular data access technologies at home and abroad,then summarized the advantages and limitations when these exist schemes in the face of the large data accessing demand and the security data storage problem.Against the background,the thesis did some research on IDA and proposed a data storage scheme,which is based on IDA and hybrid cloud storage architecture.The scheme uses Reed-Solomon(RS)erasure code to split users' data into several pieces and during the process there will generate some meta-data about how the data were divided.After data partition,the scheme stored the data pieces into public cloud storage and stored meta-data into private storage system.With that scheme,the actual data storage is separated from the meta-data storage,thus the scheme makes the file storage services decentralized and any public cloud storage platforms no longer have users' file meta-data.The scheme,in this way,not only realized the file access security and give a full play to the superiority of the cloud storage.Data accessing on the public cloud storage platforms involve the data transmission rate,data access fee,services accessibility and other issues,so this thesis also studied some data scheduling algorithms on public cloud storage service,and come up with a schedule policy to store data in different cloud storage platforms.The policy can coordinate the data access performance,cost-effective and other factors to realize the hybrid cloud storage efficiency.After describing the design of hybrid cloud storage using IDA,the thesis also presents the design of its architecture,functions,modules and implementations.The system was implemented by using a lot of open source tools.At last,we use the private cloud,which is built by HDFS and HBase to store meta-data and use public cloud service,which comprises of services provided by QiNiu,KingSoft and Ali companies to deploy the system that proposed in this thesis.After deploying the system,we tested all the system's functions.According to the test result,the system's performance,functionality is all executed as expected.Finally,the thesis drew a conclusion that the hybrid cloud storage system can satisfy the users' storage requirement transparently and steadily.