Research On Crawling Strategy Of Multi-Agent For Focused Search Engine Technology

With fast development of computer technology, artificial intelligence and computer network technology, computer technology has quickly turned to be open, network distributed and coordinative. In recent ten years, intelligent Agent and Multi-Agent System (MAS) have become a hotspot in the field of Artificial Intelligence. The advantage of intelligent Agent and Multi-Agent System broke a new path for integration, analysis, realization and application of Distributed Artificial Intelligence; it also further promoted the growth of Artificial Intelligence and software engineering.Nowadays, intelligent Agent technology has been widely applied to electronic commerce, air traffic control, workflow management, transportation systems, and information retrieval. This paper focuses on related issues, which apply intelligent Agent to focused search engines.Focused search engines need Agents to collect Web pages from the Internet by choosing to gather only particular pages related to specific topic. Comparing with the general search engines, it can effectively decrease the amount of retrieval documents, can profoundly analyze the useful topic and retrieve more high quality information for users. Therefore, focused search has become a hotspot in the search field. However, in the focused search field, most researchers focus on how single Agents to crawl the Web neglecting coordination issues. This paper concentrated on the coordination among multi-Agent in focused search engine.In this paper, a new method for retrieving Web pages is proposed. The method is based on Contract Net Protocol (CNP) and Ant Algorithm to show intelligence and cooperation in Agent. According to CNP, focused Agents are divided into three categories: Managers, Contractors, N-Agents. Managers and Contractors are the contract parties while N-Agents have nothing to do with contract. Contractors are responsible of traversing the relevant Web pages without affording energy consuming; Managers traverse the Web as well as monitoring Contractors and affording the energy that Contractors consume. N-Agents traverse the Web while affording its own energy consuming. According to Ant Algorithm, the Web pages get two properties:"food pheromone"and"visit pheromone". The higher food pheromone value a Web page gets the more relevant it is, the higher visit pheromone value a Web page gets the more frequently it has been visited recently. Agents share the information of Web pages to choose hyperlinks to follow. Finally, advantages and disadvantages of proposed Agents are discussed by experiments and conclusion is drawn: in focused Agent, adopting CNP and Ant Algorithm can significantly improve recall rate and increase the frequency of retrieving.