Data Flow In Adaptive Query Processing Mechanism

With explosive applications of information processing in sensor networks, businessprocessing, network monitoring and security, communication, industry manufacturing,and many other areas, more and more data take the form of multiple, rapid, time-varying,possibly unpredictable and unbounded streaming data (data streams) rather than finitestored data set. In all of the applications cited above, it is not feasible to traditionaltechnology on processing data, so data stream processing technology becomes a new andpopular topic in database research area.Query processing technology is the uppermost technology of data steamtechnologies. Continuous queries are the usual queries over data stream whose answersare continuously produced over time unless stopped artificially. During the run time of aquery, the execution environments change continuously, such as occupied CPU, availableRAM and so on. Meanwhile some properties of data stream itself change too, such asinput rate and selectivity of predication. Aiming at these variational factors aboutenvironment and data stream itself, in order to improve query efficiency and systemperformance, the adaptive query processing technologies emerge as the times require andbecome a key of data stream query processing technologies.This paper researches deeply on the adaptive query processing technology in datastream system and presents an adaptive cost model based on output rate—OutrateModelaiming at the continuity of queries and variability of system environment and date itself.OutrateModel treat changed input rate, predication selectivity and execution time ofoperators as its parameters, so it can continuously adapt to multi variable factors ofenvironment and data stream itself. System can dynamically and adaptively adjust queryplan to improve query performance by OutrateModel.Aiming at different operators in a query, this paper give the concrete method tocalculate the cost of every operator in a query by using abstractive OutrateModel,especially to the complicated join operator, this paper do detailed discuss. According toOutrateModel, this paper discusses different implement modes (Hash Join or Nest Loopjoin) and adaptively gives the optimized join algorithm in different executionenvironments.Generally speaking, a query often contains multi operators. On the basis of thecalculation of single operator, this paper analyzes and designs distinct adaptive optimizedalgorithms aiming at different types of query. According above algorithms, we cancompare output rate during several limited query operator ways. The query operator waywith the maximum output rate is the most optimized query operator way. In addition, thispaper can confirm when is the adaptive optimize occasion during in a time limit in orderto avoid frequently computations. So the time to transfer adaptive optimize algorithms issaved and the flexibility of adaptively is improved.This paper gives corresponding experiments, experiments proved that OutrateModelcan well simulate every operator's execution;OutrateModel can balance memory andtime latency;OutrateMode can not only optimize the time at which the last result tupleappears, but also optimize for the throughput of answers computed at any specified timeafter the query evaluation.