Design And Implementation Of Distributed Database Log Management Subsystem Based On Virtual Partition

In the distributed database based on a separate computing and storage architecture,the computing node is responsible for advancing the transaction.And after the storage nodes receive logs from the computing node,they provide the computing node with the data pages they need,which can reduce the workload of the computing node to improve the performance of the system.The virtual partition technology can flexibly manage database data and improve the scalability of the database system.When the virtual partition technology and the computing and storage separation technology are combined,the new log management subsystem is needed to perform unified and effective management of the log of the database system.By analyzing the needs of distributed database log management based on virtual partitions,a log management subsystem is designed.Its main functions are log splitting of multiple partitions,data page generation,and determination of failure recovery point.The main purpose of log splitting is to split the complete log generated by the computing node into partition logs suitable for virtual partitions;the main function of data page generation is to generate data pages for storage nodes based on the log in units of partitions;The function is how to determine the recovery point of the storage node when the computing node or the storage node fails and the memory data is lost.After that,the log after analysis and analysis can still maintain the correctness of the data page generation and the consistency of the overall data,as well as the correctness of the fault recovery point.An experimental scheme was designed and verified for the implemented prototype system.The experimental results show that the log management subsystem can perform correct log splitting and data recovery.At the same time,compared with the distributed database that does not use virtual partitions,it has a certain improvement in performance.