Design And Implementation Of WEB Of Things Search Engine Based On Hadoop

Sensing device search is a key part of Internet of Things development road, is the premise of IoT applications, yet there is no mature solution. But the traditional search engine technology for the Internet is already quite mature. Reuse of these technologies for the development of IoT search is undoubtedly a boost. In addition, the open-source distributed computing platform Hadoop with ease of expansion, mass data storage and super computing power is regarded by industry and academia as the most important tool in big data processing. In addition, Hadoop can be widely deployed on cheap PCs, which can reduce production costs. In this context,this thesis designs and implements a "Web of Things search engine based on Hadoop",which contains two core components: the query expansion module and the distributed inverted index system.The main contents of this research work are as follows:In order to overcome the traditional search engine technology low recall problem caused by mechanical symbol matching based on keywords,the query expansion module references an existing IOT Ontology, carries on the semantic reasoning based on rules, completes the expansion of the ontology to obtain the implicit information in the ontology, and extends query based on the extended ontology. Ontology correlation theory, rule based ontology reasoning and Jena are studied, the rule base for ontology reasoning is constructed, and a query expansion algorithm based on ontology is presented.Distributed inverted index system adopts the local inverted index organization strategy based on document partition. The work principle of Internet oriented traditional search engine, HDFS, MapReduce, open-source core search library—Lucene and two organization strategies of distributed inverted index are studied.In order to embody the spatial character of the IoT search, this thesis adds a distance based search results sorting module. The ontology based query expansion module, the distributed inverted index system, the distance based search results sorting module and the user query interface together constitute a complete WoT search engine.Finally, the feasibility and effectiveness of the scheme are proved by testing.