Optimization Of Time Series Data Storage Technology Based On Dynamic Downsampling Strategy

In recent years,the amount of high-resolution time series data in real-time or high-performance computing scenarios has increased significantly.These time series data needs to be saved stably,to avoid the wrong analysis results.The non-volatile memory(NVM)can provide time-series data persistent storage.However,frequently writing high-resolution time-series data to NVM will rapidly accelerate the lifetime loss of NVM.At the same time,the dynamic time warping(DTW)algorithm as a commonly metrics for quantifying the differences of time-series data,due to its elastic matching,is more and more important in the context of the explosive growth of time series data.However,the high-resolution time-series data will slow down the DTW analysis process,which makes the analysis storage process of time-series data more challenging.Therefore,it is important to design a storage optimization technique for time series data with NVM as memory.In order to achieve efficient and persistent time series data storage and analysis,this paper uses the byte-addressable NVM(such as phase change memory)as the main memory.Based on the characteristics of time series data and the disadvantage of DTW algorithm,this paper proposes a dynamic downsampling strategy for the time series data under the framework of approximate dynamic time warping(ADTW).The main contributions are as follows:Firstly,to solve the storage problem of time series data with high sampling resolution,this paper proposes a strategy,called dynamic downsampling.By using the smoothing strategy to filters out the small noises,critical point extraction reduces the sampling rate and the linear interpolation strategy to ensure the integrity.The downsampling strategy can downsample the time series data without seriously affecting the accuracy of the results obtained by the DTW algorithm,the storage overhead of high sampling rate time series data is significantly reduced.Secondly,in order to enhance the performance of the dynamic downsampling strategy,this paper proposes to use the relative change and the slope change between adjacent data points to remove non-significant data points,so as to improve the accuracy of the dynamic downsampling strategy.In addition,this paper proposes to use the high-order interpolation to improve reconstruction accuracy.Thirdly,to enable the high-performance analytics of high-resolution time-series data,this paper proposes a novel memory–storage architecture,where byte-addressable NVMs such as PCM are used as both the main memory for in-place DTW analysis and the persistent storage.Based on the storage architecture,this paper discussed various techniques to efficiently organize time-series data on NVMs.Finally,this paper conducts a series of experimental studies on publicly available time series datasets(such as ECG,Air-Temperature,Audio)to verify the effectiveness of the dynamic downsampling strategy.The experimental results show that the dynamic downsampling strategy can achieve over 90% compression rate on most datasets;in the worst case,the compression rate can reach more than 80%.In addition,the dynamic downsampling strategy proposed in this paper is more accurate than the key point strategy proposed in the earlier research,and the dynamic downsampling has less CPU overheads.