Research Of Distributed Consistency Protocol For End-Edge-Cloud Hierarchical Storage System

With the number of the intelligent devices in the Internet increasing rapidly,the Internet will produce a large amount of data that needs to be processed in real time.The traditional end-cloud architecture adopts a centralized management mode,which not only has large network bandwidth overhead but also is prone to congestion.Therefore,the end-edge-cloud hierarchical storage system based on edge computing is proposed to reduce the service delay and the communication overhead of the entire network at the cost of redundant storage of edge clouds.Replication technology ensures high fault tolerance and high availability of the system by redundant storage on multi-servers.However,the system is unreliable if the value of date stored in different copy storing is different.The existing work solves the data consistency by using the consistency protocol.Actually,almost all consistency protocols is designed for the end-cloud architecture,hence it cannot be directly used in the end-edge-cloud hierarchical storage system architecture.In addition,the classical consistency protocols are unable to adaptively meet the various needs of the application because of the fixed level of consistency.hence,design a adaptive consistency protocol for the end-edge-cloud hierarchical storage system is a very meaningful and challenging issue.For solve the above problem,we optimize classic consistency protocols and design synchronization strategies to reduce the system delay by analyzing the characteristics of the end-edge-cloud hierarchical storage system based on ensuring the high consistency of the system.Aiming the problem of the fixed level of consistency in the consistency protocol,we propose an grouping scheme to select the optimal main group for each request,which contains defining metrics for system nodes,prioritize requirements and design grouping algorithm.In addition,two different improved data consistency protocol have be proposed by combining the grouping scheme.On the one hand,the decision condition of the Multi-Paxos protocol is modified by the grouping technology,and the GM-Paxos protocol reduces the response delay and improves the throughput.On the other hand,we combine the EPaxos protocol with grouping technology to reverse modify the decision conditions.The improved GEPaxos protocol mitigates the delay burst of the EPaxos protocol at high autocorrelation rate,while ensuring the high consistency of system to maximize the system availability.Therefore,based on the characteristics and requirements of the end-edge-cloud hierarchical storage system,we choose GM-Paxos as the consistency protocol for the edge cloud according to the high availability requirement,and we GM-Paxos as the consistency protocol for the central cloud according to the high consistency requirement.Besides,we also design synchronization strategies to ensure the consistency of the data cached by the edge cloud and the centralized cloud.Experiments show that the consistency protocol of the end-edge-cloud hierarchical storage system proposed in this paper can improve the availability of system as much as possible while ensuring that the system is consistent.