Research On Ceph Heterogeneous Storage Optimization Mechanism

With the continuous updating and development of applications such as the Internet of Things,the mobile Internet and the Internet in recent decades,the data has grown exponentially.In the face of increasing data volume,the traditional single-point storage has not been able to meet the needs of the new era.In the context of new big data,the distributed storage is an effective solution to massive data storage.Ceph has attracted wide attention because it supports multiple storage services,not only provides file storage services,but also provides object storage and block storage services.At the same time,it overcomes the single point failure problem and has good scalability.However,Ceph's multi-replica mechanism does not give full play to its cluster advantage in read and write performance.At the same time,under heterogeneous storage structure,existing storage strategies also limit the performance of Ceph storage cluster.Therefore,the storage mechanism of Ceph is deeply studied and the corresponding optimization strategy is proposed.Specific research work is as follows:1.The optimization mechanism of Ceph heterogeneous storage is studied.Firstly,a write strategy based on the weak consistency of replicas is proposed to solve the problem of high write latency in the whole cluster caused by Ceph's strong consistency.Furthermore,in order to effectively utilize the I/O performance of slave replica node,a composite storage optimization strategy based on the weak consistency of replicas is proposed.The experimental comparison between the Ceph native strategy and the optimized storage strategy shows that the combined storage optimization strategy based on weak consistency of replicas improves greatly in write throughput,random read throughput,sequential read throughput,SSD hit rate and write latency reduction.2.Based on the heterogeneous storage architecture of SSD and HDD,a combined storage optimization strategy is proposed to make full use of the performance characteristics of different media storage devices in heterogeneous environments.By establishing an object heat model,a heterogeneous replica combination storage strategy based on the heat partition model is proposed.Thesis compares the cluster reading efficiency and the read-write performance of different combinations of heterogeneous replicas based on the thermal partition model.The results show that the heterogeneous replica combination storage strategybased on the thermal partition model has great advantages in giving full play to the performance characteristics of heterogeneous replicas.The research results show that under the heterogeneous storage environment of SSD and HDD,the storage mechanism of Ceph can be optimized and the access efficiency of Ceph cluster can be effectively improved by using the weak consistency strategy of replicas and the combined storage strategy.