Building A Multi-Data Center Storage System Based On Distributed Consensus Algorithm Raft

In recent years,with the large-scale application of new technologies such as Big Data,Cloud Computing and the Internet of Things,the amount of data storage has exploded.In order to meet the growing demand for data storage,various industries have begun to build enterprise data centers,improving business efficiency and ensuring uninterrupted service.Data centers have become an integral part of enterprise operations.Compared with traditional centralized data center,distributed data center has stronger horizontal expansion capabilities and can implement distributed deployment according to business requirements,which can better help enterprises to provide services.However,the construction of distributed data center needs to face many challenges,such as complex system architecture design,data consistency,maintenance management and so on,among which the most basic and important one is data consistency,which is the key to ensure the normal operation of services.In the field of distributed storage,distributed consistency algorithms are usually used to achieve distributed consistency.At present,there are two most widely used distributed consistency algorithms: Paxos algorithm and Raft algorithm.Compared with the classic Paxos algorithm,Raft algorithm is easier to understand and implement.However,due to the strong leader characteristics of the Raft algorithm,it has limitations in horizontal scalability and is not suitable for the scenario of massive data storage.Therefore,aiming at this defect of the Raft algorithm,this paper designs a Scalable-Raft distributed storage scheme.Scalable-Raft is a multi-data center storage system based on the Raft algorithm.It supports high concurrent access and massive data storage,as well as has good scalability.The main work of the dissertation is as follows:(1)Comprehensive understand the related technologies in the field of distributed storage.Focus on data distribution and implementation of distributed consistency,combine with Paxos algorithm and Raft algorithm to conduct in-depth study of distributed consistency,and understand the two algorithms operation mechanism and their advantages and disadvantages(2)The implementation and optimization of Raft algorithm.In order to improve the efficiency and stability of the raft algorithm,we need to understand the working principle of the raft algorithm,complete the implementation of the raft algorithm,and give the optimization solution for the problems existing in the engineering application of the Raft algorithm.(3)The design and implementation of Scalable-Raft scheme.This paper analyzes the limitation of Raft algorithm in horizontal scalability,and puts forward a new Scalable-Raft scheme aiming at this limitation.Scalable-Raft system supports flexible scaling and has good horizontal scalability.In addition,the Scalable-Raft scheme supports high concurrent access and massive data storage.(4)The implementation of load balancing.This paper puts forward the corresponding data load balancing strategy and Leader load balancing strategy in Multi-Data center storage to ensure that the resources of each server are fully utilized in order to obtain the best system performance.In order to evaluate the performance and effectiveness of the Scalable-Raft scheme,a multidata center storage system was built based on the Scalable-Raft scheme.The performance of the Scalable-Raft and Raft algorithm was compared through a number of experiments.The experimental results show that in high concurrent access scenarios,Scalable-Raft solution has better performance than Raft algorithm,and Scalable-Raft solution can have a good scalability,coping with higher concurrent access scenarios as the system expands.