Design And Implementation Of A Transaction Management For Distributed Graph Database Storage Engine

With the deep application of the Internet in various industries,many new requirements have emerged to analyze the relationship between things.Due to the large number of connection operations,the delay of traditional relational databases in processing these requirements is intolerable.In order to efficiently handle the correlation operations between massive and complex data,graph databases came into being.And due to the limited processing and storage capacity of a single machine,the graph database must be distributed and support horizontal expansion.In distributed graph databases,there are special application scenarios that require graph databases to have transaction capabilities: for example,the change of interpersonal relationships in social scenarios.Therefore,how to implement distributed transactions in distributed graph database is an urgent problem to be solved.Traditionally,the solution to implement distributed transactions is generally two-phase commit(2PC).Other distributed transaction solutions,such as three-phase commit,are essentially optimizations for 2PC.The 2PC protocol assumes that all nodes are available,but this assumption is not always true in the actual product environment.How to implement high-availability distributed transactions in the scenario where nodes are unavailable is a problem to be considered in this article.And the distributed transaction scheme in this article mainly refers to the Percolator transaction model proposed by Google.This thesis attempts to design and implement distributed transaction functions in a distributed graph database.The main tasks include:(1)Multi-version concurrency control: The multi-version concurrency control scheme(MVCC)is used to achieve the isolation level of Read Snapshot.And reduce the number of serializations by optimizing the MVCC storage model.Through MVCC,you can quickly get a snapshot of the system's data.(2)Distributed transactions: Design and implement distributed transaction modules,avoid using the traditional two-phase commit protocols.By optimizing the Percolator transaction model,the number of data persistence and remote procedure call on the critical path of transaction execution is reduced;Simplify the coordinator Execution logic to make it stateless.Use batch processing,asynchronous commit,pipeline processing and other technologies to optimize transaction execution efficiency.(3)Highly reliable fault-tolerant solution: Based on the distributed consistency protocol Raft,the Multi-Raft protocol is designed and implemented to ensure that data is reliably copied to most nodes.Multi-Raft introduces multiple Raft Groups.By distributing the shards in the system,the load of the system is guaranteed.It also implements automatic adjustment functions,such as when the load balancing is broken,it performs fragmentation scheduling and leader switching.In the benchmark test,a complete functional test and performance test were performed on the main modules of the system.The functional test results show that the system can correctly handle read and write requests and transaction requests,and can correctly deal with the exceptions during transaction execution.The results of the performance test show that the processing performance of read-write requests and transaction requests has reached the expected goal.