Temporal Query Analysis And Temporal Index Optimization Based On Apache Spark

Temporal data is common among many applications.With the growing amount of data,it is significant to provide an accessible,high-throughput and low-latency scheme to process the large temporal data.There exists some temporal databases and temporal schemes based on cluster-based computing systems.However,most of them are disk-oriented and degenerate rapidly when processing with large data.We propose this system which is based on Spark,and provides accessible and scalable temporal query scheme with large temporal data for users.Specifically,we extends Spark SQL parser to support temporal operations,to provide a SQL-like query interfaces for users.Besides,we use the index manager scheme based on Spark SQL which is proposed by SIMBA,and embed optimization strategies in two facets: global filtering and local temporal index.Depending on these optimization rules,our system achieves high throughput and low latency in several temporal operations.Evaluating experiments on temporal query efficiency and effectiveness shows this system's improving performance over original Spark SQL in different factors.