A Spark Based Semantic Reasoning Engine And Its Application

In recent years under the background of blooming knowledge graph,the size of the semantic web data has exploded.How to effectively conduct semantic reasoning on large-scale semantic web data is a difficult problem facing the researchers.Specifically,when implementing semantic reasoning on massive semantic web data,a large amount of calculation and long time-consuming are both very prominent problems,especially when conducting reasoning with complex rules of logic.The traditional semantic reasoning engine under standalone environment can't deal with reasoning tasks for mass semantic data.While lacking scalability considerations,it is hard to meet the demand of reasoning on semantic linked data which is ever growing on data scale.From the perspective of distributed computing,reasoning process of existing Hadoop MapReduce based reasoning engine is still too slow due to the lack of the optimization for reasoning algorithm in the distributed environment of network communication and disk I/O.Aiming at the problems above and focusing on distributed in-memory platform Spark,this paper research the following aspects.First design a good modular distributed reasoning engine architecture with configurable reasoning rule.Research existing standalone and distributed semantic reasoning algorithms,implement their distributed version based on the Spark framework,do optimization on the base of Spark's features.And then conduct contrast experiments between the Spark based semantic reasoning engine and the existing traditional distributed implementation.Experimental results show that the proposed system,which achieves better scalability,outperforms the Hadoop based reasoning engine.Finally,this system is applied in the field of Internet of things,adapting to the real-time and streaming scenario of semantic data processing and reasoning.