Analysis And Improvement Of Data Consistency Based On Redis

With the Internet's rapid development,traditional relational databases show their limitations when dealing with massive data storage,high scalability,high availability.Compared to relational databases,NoSQL databases do well in dealing with these new demands of the Internet because of their schema-free,easy replication support,eventual consistency,etc.Since the network partition is unavoidable in NoSQL,it is necessary to improve the consistency convergence rate as much as possible for NoSQL under the premise of ensuring partition fault tolerance and availability,according to the CAP theory.With the advantages of open-source,memory storage and high performance,Redis won a wide attention from academia and industry in the field of NoSQL.Researching on Redis databases' consistency is essential to promoting Redis' development.This thesis not only analyzes details of the Redis consistency,but also solves the consistency issues found in the analysis process to improve the consistency convergence rate.This thesis focuses on master-slave data consistency.This thesis analyzes the Redis consistency in three aspects: the Redis architecture,the implementation of the data replication in Redis and Redis data consistency maintenance.After that,this thesis summarizes the consistency problems found in the process of analysis and designs test cases to verify these issues.Two consistency issues found and fixed in this thesis:1.Synchronous data missing in replication backlog buffer.With using replication backlog buffer,the original Redis needs execute a heavy full synchronization between the master and slaves once the synchronized data in the backlog is covered.This thesis designs and implements a new replication backlog buffer that abled to avoid overwritten of synchronization data by storing the overflow data to the disk.2.A low efficiency synchronization strategy after failover.In order to ensure slaves' data is consistent with their master after failover,a new elected master must do a full synchronization with its slaves in the native Redis.If the new master's data is almost the same as its slaves which happened when they had a same master before failover,there is no necessary to do a full synchronization between them.A partial synchronization is implemented between the master and its slaves after failover in optimized policy.At last,this thesis tests the improved Redis.The test results show that the improved Redis achieves a partial synchronization which is translated from full synchronization in some particular scenarios and improves Redis' consistency convergence rate with a little slowing down write performance without influencing read performance.