Research And Implementation Of A Web Service Search Engine

UDDI registry center provides a common platform to publish and discover Web services. At present, more and more enterprises adopt Web services for enterprise business integration, and build the corresponding UDDI registry center. However, most UDDI registry centers are private. It only be used in enterprise-wide and does not release to outside. Microsoft and IBM, which are Web service advocaters, have closed down their public UDDI registry center in 2006. We can call this type of Web services as private services. However, since there are a lot of tools to quickly build, deploy, and parse Web services right now, many software vendors, open source and Web services programming enthusiasts are able to provide lots of free Web services over Internet. We can call this type of Web services as open services. Because of opening and universal nature of the Internet, the number of free Web services will be increased very quickly. Therefore, the full and effective access and management to the Web services which are scattered over the Internet has become a very important issue.To address the problem, we design and implemente a Web services search engine. The main work of this paper is as follows:(1) Through analysis of general search engines, we proposed the architecture of Web services search engine. The main data structure of each module is conducted in-depth.(2) We developed a method to use the API of Google and Yahoo to improve search efficiency and reduce the search space of Web services.(3) When the efficiency of a single-crawler is not so powerful, we proposed the idea of a distributed crawler which can achieve URL load balancing purposes. The experimental analysis proved that the distributed crawler model is feasible and useful.(4) In order to make crawlers focusing on the theme of Web services, we need to filter web pages we obtained. Here we use the vector space model and the improved calculation of TF-IDF weights to filtering web pages. The experiment analysis showed that we achieved the desired results.