Research On OLAP Query Technology Based On Distributed Memory

Massive data computing and analyzing in cloud environment are mainly batch-processing and offline at present. It is difficult to achieve online, ad-hoc, interactive analysis. Data cube is an important data model in data warehouse, business intelligence, according to the roll-up and drill-down relationship between data units the lattice structure of data is constituted. In order to improve the performance of query analysis, this article is based on data cube lattice model and distributed memory architecture to research efficient on-line analytical processing technology.This paper mainly researched in the following two aspects:(1) Regard lattice data structure as graph data structure, using statistical characteristics and law of lattice structure data as the breakthrough point, by using the statistical method, the complex network experiment, such as classical analytical model, the concept of hierarchical concept lattice, to reach the lattice structure of the data model; Based on this, Combining the current graph partition technology, research lattice structure data divide and store in multiple nodes, which consume less communication cost and make the cluster load balancing.(2) Hierarchical closed cube is an extension of the closed cube model and a semantic compression of data cube which can effectively reduce the storage space of data cube. Data cube can be stored by array and lattice structure. By using hierarchical information of each closed unit and cover relationship between closed units, this paper reaches such two kinds of structure of distributed storage, query method.Eventually build a distributed computing framework based on Spark which is an in-memory computing structure to implement an OLAP queries prototype, also complete experimental verification and analysis.