Design And Implementation Of Distributed Consistent Core Platform

With the rise of containerization and isolation of cloud systems,there are more and more distributed systems supporting containers.In a container-based cloud system,system resources are usually registered and allocated by a resource scheduling platform.Resource scheduling platform is faced with a wide range of problems.This thesis focuses on its data management related issues,to provide support for data storage.It mainly includes: how to design and implement the consistent strategy,so that the system can provide consistent storage services in most cases to meet the requirements of high concurrency,high reliability and high frequency data operation.Secondly,how to ensure the load balance of storage nodes in order to obtain better service performance under the existing conditions is discussed.Finally,how to quickly rejoin the cluster after the nodes are restored will make the system more available.This thesis designs and implements a distributed consistent core platform,which will be applied to the resource scheduling platform to provide support for data storage.The main work of this thesis is as follows:1)Consistent strategy.In order to make the system have better reliability and fault tolerance,and to maintain the consistency of node data in the cluster under abnormal circumstances,this thesis adopts RAFT protocol,in which the leader node is elected within the cluster to provide read and write service,and the consistency of stored data is guaranteed through log replication among nodes.In order to avoid the possible inconsistency of data returned due to cluster "brain splitting" during read operation,the leader lease mechanism was introduced,proxy leader was adopted as the external receiving request node of the cluster,and distributed lock was designed to guarantee it.2)Central node and task scheduling strategy.Can load balancing in order to make the system of data distribution,this thesis uses the consistent hashing scheme based on resource utilization,will carry data mapping virtual node distribution on the hash ring,by the monitoring center node load of each physical node according to the physical node load dynamic adjustment of virtual nodes,in order to maintain the system of load balancing.At the same time,in order to avoid the central node becoming the weak link of the system availability,a consistent design strategy is adopted to ensure its fault tolerance.3)Fast fault recovery strategy.In order to speed up node recovery,the system adopts the way of log speed up,periodically snapshot the version of the data.When the node recovers,it first loads the most recent snapshot version based on the log record,and then begins to replay the operation log.Finally,functional and performance tests are carried out on the conformance core platform.From the experimental results,the system can ensure the correctness and consistency of data,and through the corresponding optimization strategy,the performance of the system can be improved to a certain extent.