| With the development of the Internet of Things,the increase of data brings many challenges,so it is very important to develop a scalable and efficient architecture for storing,querying,analyzing and visualizing spatiotemporal big data.Non-Volatile Memory Express(NVMe)-based solid-state devices offer unprecedented performance in terms of latency and peak bandwidth.However,the complex I/O stack of the kernel prevents the full performance of NVMe SSDs from being utilized.Application-specific optimizations are also difficult on the kernel due to its generality and fairness.In response to the above problems,this paper designs a user-mode NVMe storage software stack based on the open source storage software development tool SPDK released by Intel,and develops a single-machine storage engine software for NVMe Store to give full play to the low latency and high I/O transfer rate of NVMe.Features.The SPDK NVMe vBdev block layer can provide 1.5 million IOPS on a single CPU.Under random read and random write workloads,the latency is 17% and 21% lower than the Linux Kernel Libaio latency,and 3% and 13% lower than the Linux Kernel io＿uring latency.Use Redis distributed database to deal with high concurrent read and write,use hash linear expansion chain to manage read and write metadata,and use the characteristics of cache database to add buffer layer to data.When the write repetition ratio is greater than 0.7,the read and write rate can be improved,and the read and write I/O load of NVMe can be reduced at the same time,laying the foundation for the development of future distributed architectures.The study found that when the high-frequency access is relatively concentrated,the deletion strategy according to the reading frequency performs well,and when the high-frequency access is not concentrated,the LRU algorithm deletion strategy is better.The test results show that the NVMe Store storage software has shown good performance in the read and write of spatiotemporal big data. |
PDF Full Text Request