The Design And Implementation Of An Enterprise Object Storage Gateway Based On Ceph

With the rapid development of the industrial Internet,more and more government and enterprise units have begun to introduce private clouds into their business systems,so data storage services have become very important.Object Storage,as an important storage method in data cloud storage,has also been favored by many enterprises.Develop an object storage service to meet the basic storage function requirements of users,improve storage resource utilization,reduce user costs,provide better cluster management and maintenance capabilities,and provide greater availability and stability while significantly improving the storage performance of the cluster.Through the analysis of products and technologies of cloud object storage service providers at home and abroad,it is proposed to develop a Ceph-Based enterprise object storage gateway.The problems associated with Ceph Object Storage in enterprise private cloud operations,such as: severely insufficient cluster performance in multiple scenarios,large impacts by the fault cluster,difficulty in operation and maintenance,unreasonable use of storage resources,and inability to meet some functional requirements.According to the idea of divide and conquer,realize the small-scale clustering,called Set model,which is managed by the object storage gateway layer.According to the characteristics of different storage media,tiered storage is implemented,and storage characteristics of different tiers are different,and data balancing based on a consistent hash ring is implemented between clusters of the same layer,thereby fully utilizing storage resources.By implementing the clustering of the storage gateway,the single point problem is solved,and the performance is significantly improved,especially in the scenario where the cluster inventory data is huge,and the number of single bucket objects reaches the limit.The storage gateway adopts the Set model and implements routing control.It can continue to provide high-performance storage services by downgrading the failed cluster to read-onlymode and can achieve non-intrusive exploitation of existing clusters.Since the cluster size in the Set mode is small,the operation and maintenance operation are simple and rapid.The index of the object is raised to the storage gateway layer,which is compatible with the standard protocol of S3,so that personalized functions such as metadata retrieval and analysis,and heterogeneous S3 storage clusters can be realized.At present,the development and productization of the object storage gateway has been basically completed,and basic automatic deployment,operation and maintenance,and management have been realized.In the long-term,high-concurrency online pressure measurement environment,the object storage gateway service can still provide stable high-performance,high-availability,easy-to-maintain enterprise-level object storage services.