Research And Implementation Of Large-scale Context Stream Processing Framework

With the development of mobile Internet, mobile devices, especially the wide useof smart phones, a variety of sensor data can be collected through these mobile deviceswith different kinds of sensors. Thus, group awareness based on the mobile devicesbecomes possible, which promotes its applications in various fields, such astransportation, healthcare, environmental monitoring and so on.In mobile group awareness applications, a large number of mobile devices willfirstly continually collect a variety of context information, and then process these hugeamounts of information in real time to get the state of the entire system whileresponding to user’s request with better service in a timely manner. Accordingly, how toprocess the large scale of stream information of context data poses great challenges inmobile groups aware applications. In this paper, we conduct our research based on thelarge-scale the context information stream processing technology, and present a massivecontext stream processing model based on the distributed stream computing platform S4.Further, we focus on the mechanism of the rule-based context filtering, optimizing thecontext aggregation, and the event-based task scheduling in depth. Our work includesthe following three parts:Firstly, the rule-based context information filtering mechanism is raised accordingto the large scale and numerous varieties of context information collected by mobiledevices. It combines the rue-based engine of Rete matching algorithm with the logicprocessing units of stream computing platform S4to quickly match and filter theperception of information flow described in a simple and efficient JSON format.Secondly, the context aggregation optimization mechanism is proposed toefficiently aggregate the real-time context information. Based on the Document ObjectModel, we take advantage of the combination of S4aggregation logic processing unitswith the in-memory document database to improve the efficiency in processingperception information stream by reducing the disk read/write operation times inreal-time.Thirdly, the event capturing mechanism is put forward based on task scheduling.With this event capturing mechanism and combination of group and individualawareness, individual behavior can be collected and analyzedFinally, we design and implement a real-time group-awareness oriented streamprocessing framework used for collecting group and individual behaviors. Besides, wedevelop a specific application case in the urban traffic sensing system. And theeffectiveness of the proposed framework is verified.