Research On Performance Optimization Of Erasure-code Storage

With the generation of massive data,data errors become more and more common,and fault tolerance becomes an indispensable part of storage systems.Erasure code technology is a widely used fault tolerance technology in distributed storage systems at present,which saves a lot of storage overhead compared with the traditional multi-copy redundancy technology.In order to make distributed storage system have both high performance and high reliability,memory storage with high access performance combined with erasure code technology is widely used in distributed storage system.However,cluster expansion leads to frequent node performance degradation and serious long tail latency.Although tail latency accounts for a small proportion,it becomes a performance bottleneck in distributed storage systems.In order to reduce the tail latency of memory-based erasure code storage system and improve the access performance of the system,the main work of this paper is as follows:1.The scenario of long tail latency of memory storage system based on erasure codes is designed.It is necessary to construct the long-tail latency scenario because the performance degradation nodes are difficult to occur in small-scale clusters used for experiments.In this paper,based on the analysis and statistics of the causes of performance degradation of memory storage nodes,the long tail latency scenario in memory storage system is designed by simulating the duration and slow response of performance degradation of memory nodes.2.An algorithm Short Tail is proposed to reduce long tail latency.(1)A lightweight request monitoring algorithm is designed to identify degraded nodes.The algorithm monitors the access performance of each memory node and identifies and marks degraded nodes based on the average latency of all requests within the time slice.When a node is marked as a degraded state,probe requests are periodically sent to check whether the node performance has returned to normal.(2)The method of avoiding read request and write request of degraded node is proposed.According to whether the node is in the degraded state,the node can perform the degraded read operation to avoid the read request.In order to avoid the write requests to degraded nodes,a redirection node selection algorithm with low complexity and space overhead is designed to avoid the write requests to memory nodes with normal performance to ensure fault tolerance rules and load balancing.(3)An adaptive write strategy is proposed to optimize lowercase request.This paper discovered and analyzed the relationship between lowercase request and long tail latency,storage efficiency and access performance,and used adaptive write strategy to select erasure codes or multiple copies to store data,so as to reduce the write magnification problem caused by lowercase request.3.A memory storage system with stable performance and low read/write latency is implemented and its performance is tested.The system implemented in this article uses Memcached to store and access data and is tested with YCSB and Usr?1 loads.Experimental results show that Shor Tail reduces the P99 tail latency by up to 63.77%compared with the two comparison methods.Short Tail also has significant improvements in median and average latency without affecting the system's access performance under normal conditions;Parameter selection experiments show that Short Tail is effective for variable parameter selection.