Research On Publish/Subscribe Key Technology In IoT Service Platform

With the emerging of loT (Internet of Things) and mobile Internet, the scale of Internet is extended extremely and large amounts of information are produced. Exploiting the dynamic sensing environment data produced by various digital devices, network services are growing smarter and make proactively user-centered personalized service possible.In traditional Internet service architecture, the dominating service invocation style is based on SOA (Service Oriented Architecture). The sensing-based service system is playing more important roles in the future network environment, e.g. loT service. Since EDA (Event Driven Architecture) can integrate sensing and service seamlessly, EDSOA (Event Driven SOA) is promising to be the dominating architecture in future network services. Publish/subscribe technology is a preferred incarnation of large scale distributed event driven mechanism and attracts lots of attentions from both academia and industry. This thesis focuses on the architecture design of loT service platform and some key challenges in routing algorithms and data dissemination technology under large scale distributed publish/subscribe system.This thesis made researches and contributions on following three aspects.1. This thesis proposes the loT service platform architecture under the guidance of EDSOA concepts. This platform uses a synthesis of well know techniques combined with innovation component design (e.g. routing algorithm, scalable rule matching technique). ESB (Enterprise Service Bus), WSN (Web Service Notification) and topic-based distributed publish/subscribe overlay network are combined to build up the loT service execution environment. This service platform adopts the cluster technique to construct the overlay topology for publish/subscribe system. Each cluster has autonomy to choose intra-cluster routing algorithm. The inter-cluster routing algorithm is also configurable. The whole system can support multiple routing algorithms through flexible configurations. To improve the routing performance, priority-based multi-queues routing policy is employed. To achieve fast scalable event priority determination, this thesis proposes online approximate query algorithm with O(1) time complexity. In system level features, this thesis presents platform design on high availability and fault tolerance, mobility management and service immigration, event notification service combined push mode and pull mode. Under the design guidance of this thesis, an loT service platform prototype is implemented. This prototype is deployed in some loT service systems, such as coal mine monitoring system, information management of community heating system, etc. Engineering practices prove that service platform can simplify application development efforts significantly and achieve expected effects.2. Traditional topic-based publish/subscribe overlay construction theories often assume that overlay nodes are homogeneous. To accommodate nodes heterogeneity in load balancing overlay construction for large scale topic-based publish/subscribe system, this thesis proposes the minimum idle degree model and presents formal definition of overlay construction problem (Maximum Minimum Idle Degree Topic-Connected Overlay, MMID-TCO). Then, the hardness proof of MMID-TCO problem is given:MMID-TCO problem is NP-complete and there exists no polynomial time algorithm that approximates the MMID-TCO problem within a constant factor unless P=NP. MMID-ODA (Maximum Minimum Idle Degree Overlay Design Algorithm) algorithm is built for MMID-TCO problem. The core idea is to exploit the correlation between nodes sharing the same interests and assign load according to node capacity. MMID-ODA is polynomial time algorithm. Further, performance improvement algorithm is proposed based on "divide and conquer" policy. The key idea is to break down large scale MMID-TCO problem into several small scale MMID-TCO problems by exploiting disjoint characteristics on both nodes and topics in subscription table.3. To accommodate the requirement of fast and scalable online query on event priority, one novel solution is proposed in this thesis. Traditional rule matching approaches are always not independent with rule set scale. Rule matching performance might degrade significantly as the rule set scale increases. This thesis proposes system design solution and scalable online query algorithm with O(1) performance, which can achieve fast determination of event priority on large scale rule set under distributed computing environment. Since this approach is cache friendly, system throughputs can be improved by exploiting caches on broker nodes in large scale distributed publish/subscribe system. Key ideas contain two parts. In first part, discretization method is employed to transform the matching problem between event and rule set as membership test of event signature value on rule instance set. In second part, rule instances are stored into rule database constructed with Bloom filter data structure to achieve O(1) online query performance and space efficiency. Evaluation experiments show that the algorithm is scalable and robust on different data set. The cost of this algorithm is a controllable false positive rate. In many application scenarios, a fast approximate answer is much better than a slow accurate answer.