Design And Implementation Of Cloud Disk System Based On CEPH Distributed File System

Data is one of the most central and valuable products of the Internet at today's era,individuals produce large amounts of data that require persistent storage while consuming and using Internet products.The solution of individual storage devices is no longer able to satisfy the requirements of individual users in the conditions of ever-growing of storage capacity and storage requirement,so more and more individual users become the cloud storage audience.Decades of development of cloud computing already make the storage technology of public cloud environment very mature,there were a lot of excellent cloud products had been developed in business environment,but their core technology are closed,and for this kind of public Cloud disk system which aim to serve large number of individual users,there aren't perfect and reliable overall solutions been proposed in open source community.Based on the above reasons,this thesis studies and designs the key modules and technical points involved in the public cloud disk,and puts forward a set of reliable cloud disk system implementation architecture.Firstly,the principle and technology of the remote file system operating mechanism of cloud disk are analyzed,and the CFSRPC protocol is studied and designed.The specific format and the communication mechanism of the client and the server is given.Through the implementation of the agreement,you can transparent the details of remote file operations to facilitate the client and the server to achieve the upper business.At the same time,the transaction mechanism is designed to granulate the file transfer service,and the CSDA algorithm is designed to dynamically adjust the file size of the file transfer at the protocol level,which increases the performance of the cloud disk system.Then,the core business process of cloud disk system is analyzed,and the key modules of cloud disk system are analyzed and integrated.Finally,the overall software architecture with access layer,management layer and storage layer as the core is summarized.Based on the Ceph distributed file system design core storage module,the thesis analyzes the implementation principle of CephFS and MDS and implements Ceph access module CCD according to the definition of cloud disk system architecture and CFSRPC protocol and multi-channel multiplexing mechanism.The request of the cloud disk system is divided into the file system operation request and the management function operation request,which are accepted and processed by the CCD module and the management function module respectively.The load mechanism of LVS + Keepalived in the middle makes the CCD node and the management function node support the horizontal expansion.An independent authentication system are designed and implemented according the reference of Keystone's token authentication mechanism.The client is designed with NW.js as the framework according to the definition of CFSRPC protocol.In the end,the entire cloud disk system are deployed in the test environment,then completed the test of certification process and the basic operation of the cloud disk,and a single point of the CCD performance are tested to confirm the overall system to achieve the performance requirements of the public cloud production environment.