Research And Implementation Of The Architecture Of The Distributed Meta-Search Engine Based On Multi-Agent

The existing search engine is hard to meet the requirements of the customers in the recall ratio and precision. In order to find a related content, users often need to use different search engines inquires the same content, and then integrate out the most suitable results. Meta-search engine can solve the problem very well by calling the different member search engines to search.The technology of web data extraction is the core of meta-search engine. How to extract the content from the web page accurately and efficiently is very import to a meta-search engine. In this paper, the open source web data extraction tool Web-Harvest is developed for secondary on the basis of further research. In the process, the previous page extraction way by configuration files (page selection rules) is discarded and the several key parts of the Web-Harvest is wrapped in the new way of API. In the improved way, the efficiency of extracting page is increased, and at the same time, the development efficiency of the programmer is also improved.Intelligent Agent is a very important branch in the field of computer science. The attributes of autonomy, sociality, reactive and initiative can greatly improve the flexibility and intelligence of the system. In this paper, the Agent technology is used in the information collection system. In this way, the whole information collection task can be sent to each Agent on run concurrently, so the efficiency of Information collection will be greatly improved.Meta-search engine can help users integrate search results from different search engines. It also orders query results according to the new relevance and submits them to users. In this paper, we build a meta-search engine based on MAS combined with the Agent technology, Web technology and web data extraction technology. The experiment shows that the recall ratio of the system has improved. In the search results page, the system give the new relevance of search result. This improves the user experience greatly. In terms of system architecture, the flexibility and extensibility of the system architecture are also improved. In the future, we can build more complex, more intelligent system.