High Performance Distributed Consensus Coordination Service System

Distributed consensus is a very important part of distributed theory. With the rapid development of the Internet, distributed systems are widely used and become more powerful and more complex. In many situations, we need distributed consensus coordination service system to ensure system’s high availability and data consistency. With higher requirements for availability and consistency, distributed consensus coordination service system becomes more important and attracts more people’s attention.This paper will study the design and implementation of high performance distributed consensus coordination service system and use it in our distributed memory database system-Goldfish. Goldfish is a high performance distributed column memory database system designed for OLAP scene. Goldfish can store large amounts of data and support SQL queries. Distributed consensus coordination service system is needed when we design several key modules in Goldfish system. The distributed consensus coordination service system is mainly used for metadata synchronization, metadata consistency, the configuration information storage, distributed transactions and Goldfish’s high availability.The system can ensure data consistency and high availability which are the core features of a database system. This paper will use raft protocol as the theoretical basis and design high availability scheme. The high availability scheme includes data synchronization, fault recovery, leader election, etc. In order to improve system performance, this paper designs high performance software architecture that decouples the network I/O and system’s business, reduces lock contention, and achieves high concurrency. This architecture takes full advantages of the hardware resources and improves system performance. Based on the above, this paper then describes the detailed design and implementation of important modules in the system. Command module is very important because Goldfish needs it when uses distributed consensus coordination service system. Goldfish system’s business modules can use distributed consensus coordination service system to ensure data synchronization, high availability and other features through the command interfaces.Finally, through the test, we conclude that the distributed consensus coordination service system can ensure data consistency and achieves the high availability goal. By observing read and write operations per second, network I/O and CPU usage, we can see that the system has high performance.