Research And Implementation Of High Available Distributed In-memory Database Based On Memcached

In order to improve the data reading speed of applications,more and more developers use in-memory databases in software products to improve their performance.Our laboratory also uses the Memcached database in existing products to improve product experience.However,as Memcached memory database belongs to open source software,it is not completely suitable for laboratory product requirements.This thesis researches and implements a set of memory database system MEMDB which is more suitable for existing products based on Memcached.Firstly,this thesis analyzes the deficiencies of the Memcached database used in the lab products.As an in-memory database,Memcached has good performance,however,there are also problems that the Memcached client cannot automatically update node information,load balancing does not take into account differences in data node memory capacity,and does not support persistence.In order to solve these problems,the MEMDB memory database system is designed.MEMDB system is composed of monitoring node,client and data node.Secondly,three parts of MEMDB system are designed and implemented in detail.In the first place,as the brain of the system,the monitoring node is responsible for maintaining the running status of the whole system and saving and updating the topo-logical information structure of various clusters of the system.The monitoring node uses a cluster policy to prevent single points of failure.Using the Paxos protocol to select a node in the cluster as the master node.In addition,the master node maintains the consistency of the information on the other nodes in the cluster and uses the heartbeat technology to detect the operating status of each node in the system.Then,the client provides the application with an API to operate on the MEMDB system,and it is responsible for load balancing and read-write separation of the system.What's more,the client obtains the data node topology information and copy placement rules from the monitoring node and maps the relationship between the data object and the data node with CRUSH algorithm.By storing multiple copies of the same data in the system,the client will realize the data read and write separate operation,so as to improve the overall performance of the system.Last but not least is the data node.The data nodes use Memcached's memory allocation mechanism.The LRU₂ algorithm is used to improve Memcached's memory elimination algorithm,to reduce the impact of frequent swap out of hot data to improve the hit ratio of hot nodes.At the same time,the data node makes use of synchronous replication strategy to ensure the consistency of data between master node and replica node.It saves the user operation memory data command in the way of log in the disk,and periodically reconstructs the persistent file,so as to control the persistent saved content availability and file size.Finally,according to the design and implementation of the MEMDB system above,it is deployed in the laboratory cloud platform.Then the MEMDB system is tested and analyzed in detail.The good news is that the test results show that the entire system meets the original design requirements and has strong usability.