Research And Implementation Of Multi-dimensional Data Index Structure For Meteorological Field

With the rapid increase of data volume,it is becoming more and more difficult to meet the real-time and accuracy requirements of data processing in many fields.Because data query is the basis of data processing,how to improve query efficiency is very important.Taking meteorological data as the main research object,with the development of meteorological services,the user's query demand has been increased,and the characters of meteorological data which is stored in a distributed cloud,are large scale and multiple dimensions.Most cloud storage systems store data based on key-value.This method mainly supports efficient queries of primary key and cannot support non-key efficient queries.Multi-dimensional and complex queries still need to be scanned for the entire dataset,so that the query efficiency is low.Therefore,in the key-value storage mode,the use of multi-dimensional data index to improve query efficiency has become one of the key research topics in the current academic and industrial circles.In this paper,in order to solve the problem of multi-dimensional data query in key-value storage mode,we propose an efficient MOTree multi-dimensional indexing structure for the typical data characteristics in the meteorological field.The index structure of MOTree is different from the traditional index.It is an orderly balanced tree that supports stable dynamic updates and multiple query operations,including Boolean queries,point queries and range queries.The prefix rules are adopted for index construction and querying to improves the query performance of the index.It improves the spatial utilization of index and reduces the multi-dimensional query of index.Furthermore,we design and implement the algorithms of building,querying and updating MOTree.The construction algorithm is the basis of other algorithms,including three steps: path direct insertion,path merging and redundant path removal.In this paper,we also define a multi-dimensional meteorological data query language(SMDQL),which is mainly used to preprocess and segment the user query requests.Based on MOTree,the multi-dimensional meteorological data query engine(MMDQE)is designed and implemented,which can support the construction of multiple meteorological data dimensions online index construction and the efficient query of various meteorological data.In this paper,multi-dimensional meteorological data sets and random data sets are used to compare MOTree and the existing index structures in the data dimension,the number of nodes,and the query time.The theoretical analysis and experimental results show that MOTree supports efficient point query and range query,and the time overhead of index construction and update is smaller.The index can provide efficient query for multi-dimensional meteorological data.What's more,it can satisfy more users' query requirements and has important significance for promoting the development of the meteorological field.