Snapshot And Durability Support For In-memory Databases

The increasing core counts and memory volume open opportunities to perform online transactional processing(OLTP) in memory, leading to a new type of databases called in-memory databases.The commercial availability of hardware transactional memory(HTM), which provides hardware-maintained read/write sets and automatic con?ict detection, puts forward a challenge on how HTM can successfully support concurrency control for fast in-memory transaction processing. The analysis in this paper shows that solely using HTM to protect a database transaction leads to poor scalability due to the high probability of aborting transactions. To reduce HTM-protected transaction regions, this paper proposes to leverage the theory of transaction chopping with several workloadinspired optimizations, including recovering commutativity with deferred execution,identi?cation of non-atomic transactions and a database cache for database operations,to reduce HTM aborts.While transaction chopping effectively reduces HTM aborts due to memory access con?icts, it also creates a new challenge for read-only transactions, which can no longer observe a consistent database snapshot. To address this problem, this paper proposes a lazy snapshot mechanism that constructs a database snapshot during execution which eventually becomes consistent.While providing efficient transaction processing, the volatility of memory makes durability and failure recovery a critical issue for in-memory databases. To tackle the problem, this paper proposes a durability support that offers efficient logging and fast recovery. It uses an asynchronous logging scheme based on epoch to reduce impact on normal database operations. Only deltas of database records are logged to reduce log size. Also, it provides a checkpoint mechanism with relaxed consistency on database records, and leverages multiple cores to process log in parallel to accelerate recovery.We implement the proposed techniques in DBX, and the system shows good performance and high scalability in the evaluation.