Heuristic Focused Crawling Based On Link Context

With the rapid development of the Internet, the traditional generic search engine technology has encountered some difficulties, especially can't satisfactory handle the subject queries submitted by users. To tackle this difficult problem, the topic focused crawling technology came into being and occupies a position with its high degree of specialization and objectives in search engine research and development of the next generation search engines. In addition to construct professional search engine, it can be used in a client-based real-time information retrieval system, and has become a new research hot in network information retrieval field.In response to various enquiries users may submit, the crawling technology researchs how to use online real-time selective procedures in downloading, efforts to avoid downloading irrelevant pages, as much as possible with the collection of information relevant to the submitted topic. Substantial research shows: the management strategy design of the crawling frontier will greatly affect the real crawling procedures'performance. Impact the frontier management strategy a number of factors, such as the content of the information page (page title, description and text), address information, links to information, brother and father links information, which also have been in a more in-depth study. Currently, people at the page block field, have turned attention to what influence the color and font information of the text node have on the crawling on an HTML page.Instinctively, the text surrounding a link or the link text on the HTML page is often a concise summary of the target page. Altough there was early research in the focused crawling field, the development has been contrained because of its sparse content. To give full play to its guidance, it's a good idea to expand its content scope, but if the practise is with excessive manual...