Research On Online Query Methods Of Multidimensional Metadata In Storage Systems

The development of information technology has led to explosive growth of digital data. The scale of storage systems has reached EB level with billions of files. That makes it extremely difficult to find and manage files in large-scale storage systems, so the demands for complex metadata queries are growing. While existing data storage systems based on hierarchical directory tree can not provide the functions of complex metadata queries and support the scalability for exponentially growing datasets.This paper designs and implements an online query system of multidimensional metadata, called RCube (R-tree based Cube). It provides complex metadata query functions, and is able to make quick response to some complex queries (such as range-sum and range-max query), which can help users and administrators to efficiently find and manage files.The key idea behind RCube is to view the multidimensional metadata of files as multidimensional data cube. For complex queries, it can save time of online computation and significantly reduce query latency by full or partial precomputation of the data cube. RCube uses an R-tree as its indexing structure. R-tree as a height balanced tree and multidimensional index structure, is very suitable for establishment of multidimensional indexes for file metadata. It can return the result of precomputation quickly to improve queries efficiency. RCube is precomputed for range-sum and range-max these two kinds of range query. For these two kinds of range query, it can return the result of precomputation quickly to get the final result by making use of the established multidimensional index of files metadata, which can quickly decrease the search range. The RCube uses bottom-up update algorithm and updates the affected branches only, as much as possible to reduce computation and improve update efficiency.The experiments show that in large-scale storage systems RCube can support complex metadata queries with low query latency and well scalability. The performance of range-sum and range-max queries is better than current approaches that index metadata in database.