Raft Protocol Optimization Based On Dynamic Replica Sets

In modern distributed databases,a multi-replica backup mechanism is commonly used to provide fault tolerance for the system,while master-based consensus protocols ensure linearizability among replicas.However,as the workload scales up,the master node handling all read and write operations can become a single-point bottleneck.To alleviate the load pressure on the master node,read loads can be transferred to replicas for processing.Depending on the different consistency of replica reads,existing replica read schemes can be divided into two categories: one is to provide weak consistency read results directly from the replica,where read operations are only processed by the replica locally,thereby alleviating the performance bottleneck of the master node,but this is not suitable for scenarios with high consistency requirements such as bank transactions and e-commerce payments; the other is to provide linearizable read results from the replica,where read operations need to interact with the master node and face the high latency caused by the increase in communication rounds.Therefore,this dissertation aims to provide a method for local linearizable reads on replicas by bypassing the master node.This dissertation proposes improvements to the leader-based Raft protocol by adopting a replica linearizable read scheme and presents a method for linearizable read based on dynamic replica sets.This method mainly addresses three issues: Firstly,in leaderbased consensus protocols,the high load pressure on the master node often becomes a performance bottleneck for the system.Secondly,the process of sequential scanning of log items in key-value databases severely affects system throughput and latency.Finally,in the face of various exceptions in a distributed environment,a robust fault tolerance mechanism is required.Based on these three issues,the main contributions of this dissertation can be summarized as follows.(1)Based on the definition of log recency,an dynamic replica set strategy is proposed.Different replica sets are divided based on log recency.Replicas with log recency can directly provide linearizable reading results from the local node without going through the master node,allowing read operations to be shifted to the replica and relieving the bottleneck of a single point.(2)Optimization strategies have been designed for hierarchical pull-based write and parallel scan counting Bloom filter-based read.Firstly,by adopting a hierarchical strategy,some write operations are transferred to replicas,and a strategy of actively fetching log entries by replicas is used to increase the throughput of system.Secondly,a counting Bloom filter is constructed for log entries to accelerate the speed of target key queries.Finally,the scanning process is accelerated by parallel scanning of log entries in different states to reduce the latency of read operations.(3)A fault tolerance mechanism with dynamic replica set adjustment is proposed for different failure scenarios.This dissertation lists potential failure scenarios in distributed systems and designs a complete fault-tolerance mechanism by periodically detecting and dynamically adjusting poorly performing replicas.Secondly,the correctness of the strategy is verified by TLA+ model checking for role switching.At the same time,by analyzing examples that violate linearizability,the dissertation provides guarantees for linearizable reads on replicas.In summary,this dissertation proposes a method for achieving local linearizable read operations on replicas in the Raft algorithm.By categorizing replicas and utilizing those with the most up-to-date log entries,linearizable results are provided for read operations.To further improve system performance,specific optimization strategies have been designed,such as index optimization based on counting Bloom filters,parallel reading,hierarchical writing,and proactive log entry pulling.In addition,fault-tolerant mechanisms are proposed for various failure scenarios,and formal verification is conducted using TLA+.Finally,the method proposed in this article has improved the throughput and latency by 30% and 20% respectively compared to the classic Raft algorithm.