Adaptive Query Optimization For Cross-engine Databases

Moving databases to the cloud is a technology trend that companies are considering today to achieve more cost-effective IT management.In addition,data analysis is increasingly involving more depth and iterative algorithms that require more computing power than traditional analytic workloads.In order to fully realize economic goals,cloud database systems should be able to adjust their resource consumption to accommodate different workloads.However,traditional data warehouse architecture is not flexible enough to achieve fine-grained resource control,which severely restrains the total cost of cloud provider as well as the user optimization and maintain the desired QoS.To build data warehouse for the cloud,new architecture should be studied.In this paper,we believe that adaptation should be the ability of the database system,the main attribute that the cloud database should support,and explore a structure that separates the data management unit from the data analysis unit.Adaptation is mainly manifested by the flexibility and scalability of the database system unit and the data analysis system unit.By separating the “active” components from the “inert” ones,the new architecture leads to better adaptivity.We implement the architecture using an RDBMS and an inmemory cluster computing engine with SQL support.Specifically,we build a prototype system called Duo SQL based on PostgreSQL and Spark.The main contributions of this paper are as follows:1.Extension of relational database.Enhance the data analysis capabilities of relational databases with external computing power.2.The combined user of the distributed relational database PostgreSQL cluster and the distributed computing system Spark cluster can achieve elastical configuration and storage and computing separation.This paper validates the Duo SQL system using the TPC-H benchmark and data mining algorithms.The preliminary results show that the decoupling approach has great performance potential.