Research On Key Technologies Of Distributed Semantic Publish/Subscribe Systems

Wide use of grid computing, pervasive computing and mobile computing platforms and rapid development of Internet technologies demands the distributed systems with large-scale, loosely coupled, dynamic and autonomous requirements. Publish/subscribe paradigm, as an event-based paradigm, provides asynchronous, anonymous many-to-many message delivery, and achieves the full decouple of participants in time, space and flow. It can satisfy the demands of the current large-scale distributed computing communications, and is widely used in e-commerce systems, enterprise application integration, network monitoring, pervasive computing, mobile computing, and sensor networks.Traditional content-based publish/subscribe systems still have many shortcomings in the expression of capacity, reliability and efficiency, and cannot support complex events, load balancing, and semantic routing. So it is difficult to meet the new challenges of distributed computing. Combining the requirements of the National Natural Science Foundation of China and the National High-Tech. R&D Program of China, this thesis focuses on the key technologies of large-scale distributed publish/subscribe systems, including composite and semantic data model, load balancing, parallel semantic matching algorithms and semantic routing algorithms, and provides a high performance middleware for new generation of distributed computing.We first introduce research background and significance of publish/subscribe systems, then present a detail description about research of key technologies:data models, matching algorithm, overlay network, and routing algorithm. We also introduce the latest research of publish/subscribe systems.In complex scenarios, users need to receive a specific sequence or combination of events. Traditional publish/subscribe systems usually only support simple events, are unable to meet the needs of users. We integrates research of Semantic Web and composite events, and propose a composite event and subscription language to support semantic matching of inherited properties and classes, and to support logic and temporal operations, hense extend the expression of systems.To improve system scalability, large-scale publish/subscribe systems usually composed a number of event brokers in an overlay network. Differences in the capabilities of brokers and an uneven distribution of events and subscriptions, the loads of brokers are probable unbalanced. We use Web Ontology Language (OWL) to build semantic broker network. We define weight of ontology classes, weight of brokers, and the conception of virtual subscription, create an ontology class weighted tree (OCWT) based on the inheritance and similarity of ontology classes, and map it to the weighted broker network. Events and subscriptions are routed to different brokers which manages the responding virtual subscriptions. The ontology routing table maintains OCWT and creates a large-scale semantic broker network. Based on these, we propose dynamic load balancing algorithms:weights of ontology classes and brokers can be adjusted dynamically, and brokers were able to exchange or transfer the virtual subscriptions when overloaded. Thereby the system obtained efficient load balancing. The experiments show that, the load balancing algorithms significantly increase the capacity of the broker network, achieve good scalability and good tradeoff of network resource consumption.For the composite and semantic data model, we propose a semantic matching algorithm based on a weight-based multi-index and bitwise operation. Weight-based multi-index structure of subscriptions reduces the matching of the same triple, bitwise based semantic matching algorithm greatly improves the matching efficiency. Further, we propose two types of parallel matching algorithms:independent matching algorithm and cooperative matching algorithm, and extend the system throughput. The experiments show that bitwise based semantic matching algorithm greatly improves the speed of semantic matching, and parallel algorithms based on multi-core system greatly increase the throughput.Based on the semantic broker network, we propose a semantic routing algorithm. The ontology routing table serves as short cuts and helps to reduce the hops. A distributed composite subscription management is used to decompose composite subscriptions and collect and aggregate primitive events from the different brokers to satisfy the subscriptions and reduces the overload of the network. The experiments show that, the ontology routing table helps to reduce the average number of hops and the distributed composite subscriptions management significantly reduces the load on the network.At last, we explore the implementation of JTang Composite and Semantic Publish/subscribe (JTangCSPS) prototype system which implement the composite and semantic subscription language, load balance, semantic parallel matching algorithms and semantic routing algorithms. We also present a practical application:Constructive Information Knowledge Portal (CIKP).