Partition-based Concurrency Control In A Multi-master Database

Cloud native database has become an important development direction of database market in the future.More and more database instances are deployed on cloud platform,which not only needs to maximize compatibility with traditional database,but also hopes to make full use of cloud resources to provide high-performance,reliable and low-cost database services.Therefore,the cloud database of computing and storage separation emerged at the right moment.Currently,most commercial cloud databases adopt the design of single-write multi-read,but they can no longer meet the requirements of efficient read and write of massive data,and it is difficult to achieve dynamic expansion of write performance.Write expansion capability requires that the system can have multiple master nodes at the same time to provide read and write requests,but such design may cause data inconsistency between nodes,and then destroy the consistency of the whole database system.Multi-master database's consistency maintenance need transaction know other nodes operation after running locally,ensure the correctness of the data and the consistency through across nodes of concurrent control.However,as the number of write nodes increases,the number of transactions increases could be a lot of pressure for a single node's computing and network resources.Therefore,starting from the concurrency control problem of partition-based multi-master cloud database,this thesis deeply studied how to maintain the consistency and correctness of data among multiple write nodes in the partition-based scenario,and proposed a cross-partition concurrency control method and an adaptive partitioning algorithm.The main contributions of this thesis can be summarized as follows:(1)Partition architecture based on computing storage separation:The computing layer is decoupled from the storage layer,and the system load is divided by partition method.Independent partition transaction logs are maintained in each partition.Single-partition transactions can be executed independently and concurrently in each partition,and a logical partition is introduced to process cross-partition transactions.(2)Cross-partition concurrency control based on mixed transaction log sequence number in multi-master database:The introduction of logical partitions will ensure that only need use cross-partition concurrency control between logical partition and the rest single partitions.Collect transaction logs by transaction,assign globally unique mixed transaction log number to each transaction log,and determine the relative order of potentially conflicting transactions by comparing each field of transaction log number,so as to maintain the consistency and correctness of data.(3)Adaptive partitioning algorithm:In order to realize dynamic change of partition strategy under time-varying load and reduce the occurrence of cross-partition transactions,an adaptive partition algorithm is proposed,which can realize partition reorganization by collecting data access information of transactions and running according to batch.The above schemes are implemented in the form of plugin in the open source database MySQL,and its correctness and performance are verified by experiments.