Efficient Multiple Query Optimization System Research And Application

Multiple query optimization (MQO) in the cloud has become apromising research direction due to the popularity of cloud computing,which runs massive data analysis queries (jobs) routinely. These CPU/IOintensive analysis queries are complex and time-consuming but sharecommon components. It is challenging to detect, share and reuse thecommon components among thousands of SQL-like queries. Previoussolutions to MQO, heuristic or genetic based, are not appropriate for thelarge growing query set situation. In this paper, we propose theLineage-Signature approach to solve the MQO problem in the cloud witha recurring query set. We first generate signatures for each query based onan abstract syntax tree (AST). Then we make a simple but efficient indexfor further identifying and sharing common components of multiplequeries. We develop a sharing system called LSShare using our proposedLineage-Signature approach. Our system has been prototyped in adistributed system built for massive data analysis based on Alibaba’scloud computing platform. Experimental results on real data setsdemonstrate the efficiency and effectiveness of the proposed approach.