Research On Time Series Data Storage And Retrieval Method Based On Open-Channel Solid State Drives

In the industrial Internet of Things scenario,the proliferation of sensor devices has led to an exponential growth in the volume of time series data generated.Effectively storing and managing this type of time series data has become a critical challenge that needs to be addressed.While time series databases are commonly used for the storage and management of such data,existing solutions have limitations.Firstly,they fail to fully consider the specific characteristics of structured,stable,and predictable time series data in the industrial Io T context.Secondly,the performance constraints of traditional hardware for time series data processing have become increasingly apparent.To overcome these limitations and meet the requirements of Io T applications,new hardware solutions are required.Hence,this paper presents a design and implementation of a time series data storage and retrieval system on a real platform,leveraging the characteristics of time series data and the new hardware technology of Open-Channel SSD(OCSSD).To address the issue of existing time series databases not fully considering the characteristics of time series data during storage and management,this research proposes a storage and retrieval method for multi-source time series data.Additionally,recognizing that many storage systems built on OCSSD are predominantly based on simulation platforms,a real-world implementation of an industrial Io T time-series data storage and retrieval system,named OCTS,is developed on real OCSSD.The system incorporates redundancy elimination and caching strategies,space management strategies,and time series data retrieval strategies.Subsequently,a comparative test and analysis are conducted with the time series database TDengine.Experimental results demonstrate that OC-TS achieves up to an 8.5 times improvement in writing speed compared to TDengine and up to a 1.5 times improvement in query speed.Furthermore,OC-TS consistently maintains stable performance during read and write operations,effectively handling changes in the number of sensors.To enhance the stability of systems built on OCSSD in multi-threading and multi-sensor environments,this research proposes a multi-channel space management strategy and a multichannel ring buffer strategy.Leveraging the unique characteristics of OCSSD,the sensor data is distributed across different channels during physical space allocation on the host side.This approach mitigates the performance impact of OCSSD during effective page migration and fully utilizes the parallelism offered by multiple channels.Moreover,within the OCSSD block device driver,a single ring buffer is divided into multiple ring buffers,with the number of divisions matching the number of channels.Each buffer is assigned a dedicated write-back thread,reducing the waiting time for free space when multiple sensor data is written to the ring buffer.Subsequently,a multi-channel timing data storage and retrieval system,named MC-TS,is developed and compared against OC-TS and TDengine,time series database,for testing and analysis.Experimental results indicate that MC-TS achieves a 23.2% improvement in data writing performance compared to OC-TS and a 4.3 times improvement compared to TDengine in the multi-threaded multi-sensor scenario.Furthermore,MC-TS exhibits greater stability in the multi-threaded write scenario.