Design And Implementation Of Popularity-aware Redundancy Scheme For In-Memory Stores

With the increasing amount of datasets in the storage system,an in-memory store has become a key component for,data-intensive applications like OLTP and OLAP.However,there exists two technical challenges during the design of in-memory stores: on one hand,in-memory stores face such problems as the data loss or unavailable temporarily;on the other hand,user access patterns affects the popularity levels of in-memory datasets,whereas,static data layouts cannot match the changing access popularity dynamically.For the former,a redundancy scheme needs to be introduced to the in-memory store;for the latter,the data layout of in-memory datasets should be adjusted according to the current access popularity level dynamically.Therefore,in this paper,we study a Popularity-aware Redundancy Scheme for In-Memory Stores,i.e.PaRS.In order to achieve a good trade-off between memory utilization and access parallelism,this paper investigates a hybrid redundancy scheme for in-memory datasets,which combines both replication and erasure coding.To enable the redundancy scheme to be adjustable with the changing access pattern,a popularity-aware redundancy scheme(i.e.,PaRS)is designed for the hybrid-redundancy-powered in-memory store.PaRS can divide the hot and cold data according to the access frequency of a data block;additionally,it can support dynamical redundancy transformation(i.e.,add,delete or replace the corresponding data block).With the PaRS scheme,the in-memory store can not only respond to the user access request quickly,but also improve its memory utilization.In particular,the in-memory data is originally organized in a hybrid redundancy manner,the redundancy transformation operations allow the number of high access frequency data block to be increased to improve access parallelism,and thenumber of low access frequency data block to be reduced to improve memory utilization.Meantime,during the PaRS scheme,the popularity level is divided by mathematical model,the cluster load is optimized by various methods,the transformation strategy is defined to accelerate the completion time,the factor is added to reduce the network overhead,and the data update is designed to ensure the data consistency.We implement the PaRS scheme and two alternative redundancy schemes(i.e.,a replication scheme and an erasure coding scheme)in an actual in-memory cluster environment.Replaying the request traces generated by the YCSB benchmark,we evaluate the above three schemes quantitatively.The experimental results show that PaRS enables in-memory stores to exhibit higher access performance and memory efficiency than the replication scheme.Furthermore,PaRS achieves better node-level load balancing than the erasure coding,while sustaining superb access performance and memory efficiency.In particular,under a double-fault-tolerant in-memory store of limited memory,PaRS improves access latency by 15.1% to 31.5% compared to 3-way replication,and PaRS enhances load balancing by more than 3.9× relative to Reed-Solomon coding.