Research And Application Of RDF Stream Reasoning Based On Spark

The Semantic Internet of Things(SWoT)technology is a deeper extension of the Internet of Things.Its feature is based on semantic technology,combined with rich knowledge for semantic query and reasoning.The sensor network is its most basic facility.When different sensors continuously generate data,in order to better express and process these dynamic data streams,timely obtain valuable hidden information and knowledge in massive semantic data,understand status information and react,RDF Stream processing(RDF Stream processing,RSP)technology was proposed.The ability to perform complex reasoning on the semantic data streams generated by sensors has now become an important research area in the Semantic Web community.The difficulty of inference is that the amount of data is too large,the calculation takes a long time,and needs to meet the real-time characteristics.Obviously,it is difficult for single-machine systems to use large-scale semantic data inference.Most current RDF stream processing systems are implemented based on SPARQL(W3C standard RDF query language),but these engines have limitations in capturing complex user requirements and processing complex inference tasks.In addition,the distributed processing framework based on Hadoop faces the shortcomings of stream processing,and the efficiency continues to decrease with the increase of the amount of data.This paper mainly studies this problem.Based on the above problems and strategies,this paper designs and implements the RSP method based on the Big Data framework Spark Streaming and Kafka,combined with RDF / RDFs,OWL,ASP and other semantic technologies to jointly support the execution of the engine to achieve efficient inference of real-time semantic data ability.This article takes the Smart Home SSN(Sensor Semantic NetWork)as an example,first analyzes the common characteristics and complex events between devices,and builds a smart home ontology library based on the existing SNN ontology,and then generates RDF instance objects based on the ontology library in real time,and The RDF stream is constructed through the middleware Kafka,and then the RSP-SR engine is constructed for data query reasoning.At the same time,this paper also designs the window division strategy of RDF flow,and deduplicates,filters,divides unnecessary triplets,etc;designs the data partition model to ease the calculation work before inference;selectively conducts static based on the event mode set by the user knowledge base loading,and query optimization,etc.Finally,based on the existing evaluation methods and test indicators,compare this method with Sparkwave and S2RDF(Spark-based RSP method)in terms of throughput and memory usage.The results show that the use of Spark components(SparkSQL,Graphx,SparkML,etc.)and ASP methods greatly simplify the implementation of complex inference programs,and the iterative calculation based on memory greatly avoids the repeated reading of intermediate results generated during inference.Therefore,it is feasible to use this method for RDF Stream inference,and it is higher efficiency and scalability.