The Data Stream Adaptive Query Processing Techniques

Recently, as the rapid evolution of the network technology, a new class of data-intensive applications has become widely recognized: applications in which the data is modeled best not as persistent relations but rather as transient data streams. Because data streams are continuous, unbounded, rapid and time-varying, traditional database management systems (DBMS) is not very suitable in processing this kind of data. So the data stream management system (DSMS) is being developed to focus on data stream processing.One of the biggest challenges in DSMS is the especial need for adaptivity, due to the long running continuous queries and time-varying feature of the data stream and the network environment. This is still a topic that has not been well studied in every detail. And to date, among the different prototypes in data stream query processing, the Eddy system is the most promising technology in adaptivity. The research of this paper is based on the Eddy, focusing on the adaptivity of query processing in DSMS. There are two significant innovations in this paper that make the Eddy more efficient and adapitive than the old one. First, we improved the SteM mechanism, which was used to deal with multi-joins in the Eddy. We proposed a method that can keep the intermediate results of the join operator, without reducing any adaptivity of the original SteM mechanism. And so more compute resources are saved to improve the whole throughput of the system. Second, as the granunarity is too fine in the original Eddy, especially in the circumstances that data and environments are relatively stable, we proposed a mechanism that can adaptively changing the granunarity of the adaptivity according to the varying rate of the data and the environment. So the system is more efficient in stable circumstances, without losing any benefit of the original adaptivity in unstable circumstances. All the details of the two contributions are described in this paper and promising results are given. We also indicated the future work at the end of this paper.