Efficient And Reliable Transactional Processing Using HTM And RDMA

Main memory distributed databases has been widely used in applications such as e-commerce systems and social network applications.These applications rely on transactions to manipulate data consistently and reliably.With the emergence of advanced hardware features such as HTM and RDMA,systems which leverage them can process transactions with an order of magnitude higher throughput than traditional systems.Two core requirements for distributed systems are availability and elasticity.However,traditional protocols need to be extended or re-designed to cope with new hardware features.Thus,current systems does not support these requirements,which makes these systems not suitable for real workloads.This paper propose DrTM+br,which adds availability and elasticity support for DrTM+R,an stateof-the-art system which leverages HTM and RDMA to process transactions.DrTM+br firstly leverages a primary-backup replication schema to store data reliably.After server failures,requests which goes to failed servers are redirected to backup servers,giving users an always-on experience.To provide elasticity,DrTM+br leverages data migration to dynamically adjust data layout of the database according to the current workload.With the progress of data migration process,user's requests are more evenly distributed to servers,achieving better load balance.To ensure consistency in cases of failures,DrTM+br uses a replication protocol to synchronize transaction's output to backup servers.On server failures,some transaction's output may be left in backup server's logs,thus a recovery protocol is used to recover these transactions from backups.To migrate data without interrupting on-going transactions,DrTM+br uses a pre-copy based live migration schema.DrTM+br must not sacrifice current system's high performance.However,to ensure consistency,before a transaction's output is synchronized to backup servers,this output shall not expose to other transactions.To avoid blocking other related transactions,DrTM+br extends concurrency control so that it can detect data in-consistency and abort relevant transactions.To reduce recovery time,DrTM+br leverages concurrency to recover data in parallel.Also,DrTM+br uses RDMA to detect server failure quickly.Data migration with pre-copy has the benefits that this process can run concurrently with transaction's execution.DrTM+br further leverages existing backups to reduce the size of migration data so that data migration process finishes in shorter time.An evaluation on a six-node cluster with popular OLTP benchmarks including TPCC and SmallBank shows that,after one server has been shut down,DrTM+br can recover in 30 ms such that the system reaches it's top performance.Also,DrTM+br's data migration process has nearly no effects to system's normal execution,and it achieves shorter migration time than the state-of-the-art post copy based method.