The Research Of Hybrid Distributed Storage Scheme On Large-Scale RDF Data

With the development of the Linked-Data, RDF data sets on the semantic Web appear large-scale explosive growth. Its semantic information is more and more rich. In order to rapidly storing and efficiently managing the large-scale RDF data sets, it becomes an important problem to research new RDF data storage solutions for the RDF data management. The traditional data storage schemes are primarily based upon Hard Disk Drives(HDD). However, with the appearance of large amount of data on the Web, the read/write performance based on HDD has reached a bottleneck. Thus, the emerging of Solid State Drives(SSD) has provided an opportunity for the storage of the Web of data.In this paper, we propose an SSD/HDD hybrid distributed storage scheme, called HDStore, for large-scale data. The single fix-sized journal file using the append-only mode is stored on SSD to support efficiently read and write, while several segment files focusing on read are stored on HDD. At the same time, we use the Least Recently Swap algorithm(LRS) in the three-tier storage architecture. It uses the low-capacity SSD as a cache and takes advantage the features of SSD. When the index-sharding is swapped from memory, a series operations of build, split, move, merge are taken place on the SSD cache. It optimizes the System performance of I/O, and control the hardware cost. At last, it implements the hybrid distributed storage scheme on the large-scale RDF data.In this paper, the theoretical analysis and experimental results show that under the same hardware, data sets, such as environment, the HDStore hybrid distributed storage scheme with optimal mass RDF data loading and query performance, especially the RDF data loading performance, compared to traditional HDD based data storage solution increase about 15%.