Design And Implementation Of Multi-level Cache On Distributed Databases

In the architecture of computing and storage separation,the local cache of both the computing and storage nodes has a significant impact on the overall performance of the distributed database system.In order to cope with the shortage of cache caused by dealing with the massive data,it is necessary to provide a multi-level cache system with large capacity and low cost for computing and storage nodes to support the rapid acquisition of data information.Aiming at requiring of accessing data pages in a high speed of computing and storage nodes,a multi-level cache system architecture is designed,which integrates cache partition management,multi-level cache and monitor and adjustment,and the relevant strategies are given.The multi-level cache system consists of 5 modules,namely,the first level cache module,the second level cache module,the partition management module,the filter layer module and the monitor and adjustment module.This paper presents the partition management strategy,data page fetching strategy,cache page adding strategy and monitoring and adjusting strategy of the distributed database based on the architecture of computing and storage separation.The Partition Management Strategy of multi-level cache system uses the partition method to reduce the contention of lock resources between operations,the policies of getting data pages and adding cache pages jointly prescribe the handling of data pages in the working process of multi-level cache,the strategy of monitoring and adjusting is to adjust the capacity of two-level cache in multi-level cache system according to the real load of database system.In order to make full use of the advantages of the NVMe solid-state drives and avoid the problems of write amplification and poor write resistance,the multi-level cache system adopts the method of fixed-length storage and limited number of data pages written to two-level cache respectively.Several schemes are adopted to test the performance of the implemented multi-level cache system.The test results show that the performance of the system can be improved by the multi-level cache system both on computing node and the storage node,but the performance of the multi-level cache system on storage node is improved little,when the rate of the network transmission between the computing node and the storage node is lower or the read-write rate of the NVMe solid-state drives is higher,the more greatly performance of the multi-level cache system of the computing node can be improved,the more energyefficient of the multi-level caching system of computing nodes is.