Research And Implementation Of Elastic Cloud Storage Service In Public Cloud Environment

Nowadays,this technology era is generating a lot of data all the time.The storage requirements for PB,EB and even ZB are not difficult to find,and the magnitude of data is constantly breaking through our imagination.Individuals or businesses building their own private cloud storage service require huge up-front investment and post-maintenance costs,while public cloud storage service is more in line with the needs of most individuals and small and medium-sized businesses today.Users of public cloud storage service only need to pay a small fee to the service provider to obtain highly reliable,robust and scalable cloud storage service.Currently,cloud storage service provided by many companies require users to pay in advance to expand capacity,which is not really pay-per-use mode.In most cases,the actual usage of the user does not reach the upper limit of the storage capacity that he or she purchases,so the free storage space is wasted and the user pays unnecessary expenses.Of course,some companies provide cloud storage service with automatic expansion,pay-per-use,and local mount functions.However,due to commercial competition,the core architecture and technology of public cloud storage service are still not public.Based on the above background and laboratory project requirements,this thesis researches and designs various technical points and key modules involved in public cloud storage service,and proposes a feasible solution.First of all,in the process of system analysis,it is found that there are two problems that cannot be ignored.On the one hand,the system will store massive data for PB and even EB in the future.On the other hand,there will be a large number of duplicate files in the storage space.For the two problems,this thesis designs a storage node expansion under prediction algorithm and a file deduplication algorithm.Based on this,this thesis designs an elastic cloud storage architecture,in which the storage layer is built by Ceph distributed storage system,and the load balancing layer is built by LVS,Keepalived and reverse proxy mechanism,so as to ensure the horizontal expansion capability of the system under the overall architecture and make it able to cope with future user growth pressures.Then this thesis designs and implements five core modules of the system,including elastic expansion module,file deduplication module,pay-per-use module,background management module and user operation module.Finally,this thesis performs functional test and performance test on the system according to the system requirements.The test results show that the system meets the initial design goals and requirements,and can provide stable and efficient elastic cloud storage service in the public cloud environment.