The Research Of Focused Crawler Of Vertical Search Engine

With the rapid development of Internet technology,searching engines have become the preferred application for netizens.In order to search for more proper information,the more professional and accurate vertical searching engines have become indispensable tools for people to seek their knowledge.How to access the network effectively is the main focus for netcrawler study of searching engine.1.In this paper,relative theories of topic crawler study in searching engine are analyzed first.It studies the classification of the searching engines?the structure of general searching engines?the vertical searching engines and its four key techniques.Relative study has done on net-crawler,as well as it's key techniques,it's standards and the peculiarity of the topic crawler.2.Then the searching strategies of net-crawler in searching engine are analyzed.It studies the definition and it's classification,along with the merits and demerits of various searching algorithms.It compares the two searching engines in which one is based on content assessment and the other is based on link structure assessment.The typical algorithms of the two searching strategies applied widely are analyzed.3.The net-crawlers based on content assessment searching strategies perform well only when searching closer to relative pages,which is not overall but"short-sighted".The searching strategies based on link structure assessment,by which the structure features of link are taken into account,has the problem called "topic drift" and high computational-complexity.As the single evaluative method cannot predict the true value of URL link effectively,the two evaluative methods are combined and some searching strategies based on comprehensive value are proposed so as to improve the accuracy of the prediction of link value.From the perspective of analyzing the topic similarity,some topic-related URL is found through searching strategies based on content assessment,then the crawling order is set according to the order of URL based on link structure,so as to improve the overall searching efficiency.4.It elaborates the searching strategies of topic crawlers based on comprehensive value,and designs the model of searching strategies.Its system is realized through JAVA,and it's experimental assessment is done under the possible environment.