Optimization Of SQLite Based On Non-volatile Memory Buffer

Relational database management system is used in a variety of scenarios to ensure the correct execution of transactions and zero loss of data content.The performance of the system is often limited by the read-write performance of the persistent device.The main reason is that the performance of the persistent device is several orders of magnitude slower than the main memory in read-write delay,and the persistent device can only support read-write by block.Therefore,in the design of database management system,more consideration will be given to balance the performance differences between volatile memory and persistent devices.However,if the database is built on non-volatile memory,this problem is not so critical.Non-volatile memory has the same read-write delay and bandwidth as memory and supports byte addressing.If the database management system is deployed on non-volatile memory,the efficiency of the system can be greatly improved.However,simply replacing the storage medium will not fully exploit the performance of this hardware.Therefore,in the context of growing maturity of non-volatile memory technology,in order to better adapt to non-volatile memory,it is a very worthwhile issue to redesign and research the architecture of database management system.Based on the hardware of Intel Optane DC Persistent Memory,this paper chooses SQLite as the prototype system to discuss the related problems.SQLite is a small,fast,high reliability,fully functional database engine.The easy-to-use design makes SQLite very widely used in the practical applications.However,the disadvantage of simple design is that it chooses an expensive solution when providing transaction support.For example,in the buffer design,SQLite corresponds buffers to database connections one by one,which makes the memory space overhead large and has data consistency detection problems;At the same time,its transaction execution mode is single thread serialization,and the log record is also carried out by page,which will bring write amplification,low performance,tail delay and other problems.In order to explore the use of non-volatile memory and solve the current problems of SQLite,we build a new SQLite optimization solution based on non-volatile buffer SQLite-CC.The main work and contribution of this paper can be summarized as follows:1.We propose a non-volatile buffer organization and management scheme based on redundancy.In this paper,a dual version buffer is used to adapt to the characteristics of non-volatile memory,and the new hardware performance is fully exploited through appropriate redundancy and differential copy.On this basis,this paper uses the approximate LRU algorithm based on timestamp to optimize the non-volatile buffer.2.We propose a timestamp-based transaction management mechanism.Through the allocation of global timestamps in the buffer,this paper coordinates the relationship between the execution of transactions,and ensures the performance and correctness of read-only transaction concurrency and read-write transaction during the execution of SQLite-CC.3.We make full use of the characteristics of non-volatile memory and design a recovery mechanism.In this paper,based on the non-volatile buffer,by modifying both page index and current page record,and subtracting the log record of the original SQLite system,the problem of write amplification and tail delay in disk can be avoided.In this dissertation,we integrate the above implementation solution and optimization techniques in SQLite,an open source relational database system,and design a large number of experiments to verify the correctness and efficiency of these techniques.Experiments based on standard benchmarks such as,YCSB and TPC-C,show that SQLite-CC has 3x performance improvement over SQLite-Journal mode,and a range of 10%to 20%performance improvement over SQLite-WAL mode.