Design And Implementation Of High Availability And Scalable Redis Cluster

Redis is an open source and high-performance key-value pairs storage engine that provides a variety of data structures: string, hash, list, set, sorted set and bitmap etc.. Redis can be used as not only a cache, but also a storage. The native Redis from open source community support for clustered relatively weak, so a distributed framework based on open source community has been developed. Redis provides a distributed, reliable, simple service. Redis plans to provide official clustering scheme in Redis3.0, but so far it’s still in beta and not been fully validated in cross-regional condition, is not suitable for production environments.Under the condition of big data and high QPS and the research of the performance of native Redis instance, the article achieves the cluster by Proxy scheme. Through researching and integrating other basic services, this paper puts forward the overall architecture of the system, including the Proxy layer cluster traffic load balance algorithm, selection of the communication protocol, multiple-threaded dispatching model, the I/O multiplexing, connection pool, resource pool and multiple-threaded memory allocation performance optimization. Through analyzing the cluster Service-Level Agreement(SLA),this article model the resource of the cluster and realize the management of cluster resource. By the cluster platform, the coupling between services has been reduced. Finally, the paper user cluster management platform to deploy online services and manage them, then monitor the QPS level and service quality.On the basis of the design and implementation of the schema, the whole system includes Proxy unified access layer, Redis storage cluster, and cluster manage platform. It provides services including online access, deployment, migration, relationship management, capacity management, service observation and failover, and also provide Web user interface that can be monitor the service’s real time status from different aspect(such as QPS, time cost, key counts, memory used and timeout ratio, etc.). In this paper, the implementation of the scheme of the system has been used in the production environment and the test results show that the design can meet the needs of high I/O,high availability and good scalability.