The Research Of Stream Computing Technology Based On Data Processing

The increasing development of new-generation wireless networks bring new requirement tothe real time process of large data. RFID enables every item to access the network. Microsensorscontinously detect the environment and deliver data to the control node which is under hugepressure. Thus it is necessary to create a data processing mechanism to classify and compute realtime data stream prescisely and transform the data into benefit before its value evaporates.Therefore, the techniques of stream computing is created.The function of the stream computing is to select desirable information from raw stream dataand to process it before delivering it to the users. The characteristics are to analysize usefulinformation in the ever-changing data stream.This dissertation studies the data-driven calculation of complex events by analyzing datastream and setting up query-matching patterns. Distributed control of data is also studied toestablish a real time scheduling mechanism that assigns tasks to different operators efficiently. Atlast a socket interface is designed to improve system capacity in dealing with large connections.Main researching contributions of this dissertation include:(1) Due to the out-of-order event, the accuracy and speed of query-matching will be degraded.Therefore a real time query-matching(RQM) algorithms for generating and matching queries areproposed to raise accuracy in query-matching with relative low overhead and to shorten theresponse time as much as possible.(2) Meanwhile, a dynamic assignment scheduling(DAS) algorithm is presented to solve theproblem that traditional scheduling algorithm is less efficient when there is a large data stream. Inthe algorithm, multiple logical modules with multiple operators are setup. By transforming theoverall scheduling problem into the public sub-problem of every operator, the overhead is decreasedand the accuracy is raised.(3)To stablize the stream computing server, a server with a thread pool is setup and a supportscheme is designed to deal with emergencies. What is more, the overhead of creating extra threadout of the thread pool is calculated. At last, a scheme that uses a buffer pool to store extra threads ispresented and the system is optimized for emergencies.