Research On Multiple Aggregations Over Data Stream

Data streaming system is becoming essential for monitoring applications such as net-work intrusion detection and financial analysis at present. In data streaming system, ag-gregation is a common type of query, and the sliding window is often used. Therefore,the efective and efcient algorithms for dealing with sliding window aggregation over datastreams are more important.Data stream systems often have to process multiple aggregations, but executing eachquery separately can lead to significant scalability and performance problems; and when thesystem overload, the performance of system will appear the phenomenon of degradation andeven paralysis. The main research works are as follows:Firstly, we have analyzed the aggregate processing method in data stream managementsystem. Finding sharing resources among the multiple aggregations, it can eliminate theredundant calculations in the capability of the system; When the arrival rate of data streamsurpass the bearing capacity of the system resources, the drop operator must be insertedbased on diferent situations.Secondly, this paper presents the Optimized Paired Window Aggregation for the aggre-gations with varying time windows. According to the parameters of time windows, queriescan be classified into diferent groups, all the queries in the same group are scheduled atthe same time, and then paired windows chop a stream into unequal slices. This approachreduces the required bufer size with relatively small number of slices, and also it reducesthe computation cost by re-evaluating the similar queries.Finally, a new type of drop operator with shorter update interval called”MinWinDrop”is introduced. The system can place MinWinDrop operator before every operator at querynetwork, dropping tuples as early as possible. As a result, it eliminates unnecessary over-head. This load shedding strategy not only ensures the correctness of aggregate results, butalso maximizes the output of the queries.Experiments demonstrate that optimized algorithms can make the process of multipleaggregations more efectively, extend the aggregate applications, and have certain practicalsignificance.