Research On Semantic Web Service Discovery Based On Hadoop

In the context of big data,various industries that use the Internet as an information exchange medium are generating large amounts of data every day.With the rapid development of Semantic Web,the number of Web services is growing geometrically.As an important part of the Web service architecture,the discovery of Web services has attracted more and more attention from scientific research institutions and universities.In the face of massive Web services,how to find Web services that meet user needs more efficiently,accurately and quickly has become a challenging problem.Most Web services are described by Web Ontology language(OWL)or Ontology Web language for service(OWL-S).In this paper,OWLS files describing Web services are transformed into semantic data files of Web services in the form of triples,and then these semantic data of Web services are stored queried to achieve the discovery of Semantic Web service.This paper is divided into the following parts:By building a Spark cluster on the distributed platform Hadoop,using HBase as a database,the performance of data storage and query can be improved by taking advantage of the advantages of memory-based distributed computing of Spark and the features of HBase based on column storage and easy horizontal expansion.In order to access massive semantic data of Web services more efficiently,three tables are designed and implemented: T＿SP＿O,T＿PO＿S and T＿OS＿P,which are used to store the data of triple structure,and several ways of Spark operating HBase table data are compared to select the more efficient way to apply to this system.A City Hash coding model is proposed to encode the semantic data of Web services to be stored.By analyzing the collision rate and average execution time of several encoding algorithms,comparing the length of the encoding string,it is verifiedthat the coding method used in this paper can effectively save storage space.This paper proposes a method of transforming SPARQL query statement into HBase table data query,and designs some specific algorithms of Web Service Discovery based on SPARQL combining with several patterns of SPARQL query.A large number of experiments are carried out to test the storage and query performance of the system by using the standard Web service test sets and different scale LUBM data sets.By comparing with previous experimental schemes,the effectiveness of the storage and query schemes designed in this paper is verified,and the purpose of semantic Web service query is achieved.