A Benchmark Suite For Distributed Transactional Databases

Distributed transactional databases have recently gained popularity for their remark-able capabilities in Scalability,Availability and Schedulability.These three features con-tain the corresponding techniques,bottlenecks and optimization means of the databases.To fairly evaluate the processing ability of each database in these aspects,quantitative,unbalanced and dynamic controls as well as target-oriented evaluation metrics are indis-pensable.Despite the abundance of transactional benchmarks and tools,they are inade-quate in giving solutions for evaluating these features.They may mislead the comparing,designing and implementing of the modern distributed transactional databases if applied.To effectively evaluate distributed transactional databases,this paper designs and implements Dike,a new benchmark suite extended from the popular TPC-C benchmark,which aims to launch a fair and in-depth evaluation of distributed transactional databases.This paper combines quantitative distributed transactions and contention control to design workloads to achieve scalability evaluation; availability evaluation is achieved by building an agile exception generation mechanism to simulate hardware exceptions; schedulability evaluation is achieved through the unbalanced control of data and workloads; new met-rics are defined to unify the core design of distributed transaction processing.The main contributions of this paper are summarized as follows:1.For the first time,three new properties of distributed transactional databases are identified,along with the corresponding techniques,bottlenecks and existing opti-mization methods.The design requirements for benchmarks are presented.2.Quantifying distributed transactions and contention control is introduced to bridge the gap in existing benchmarks for evaluating scalability.3.A general exception generation mechanism is built to perform agile fault simulation,which makes up for existing benchmarks in evaluating availability.4.The unbalanced control of data distribution and workloads access distribution is designed and implemented to improve the shortcomings of existing benchmarks in evaluating schedulability.5.A customizable evaluation tool is developed to enable the definition and control of data and workloads characteristics,and clear metrics are defined for the compre- hensive evaluation of distributed transactional databases.This thesis applies this benchmark suite to Ocean Base,Ti DB and Cockroach DB for extensive experiments,obtains rich evaluation results and conducts detailed analysis to prove the effectiveness as well as usability of the tools.