Link Availability Analysis And Speculating Technology Investigation Oriented On Web Search

With the information on Web growing explosively, search engine has been the main way to seek and obtain Web information. However, not only the information on Web is huge, unstable and distributively storaged, but also the network is unstable, which leads link availability problem to search engine. The problem is defined as search results containing some links that can’t be connected or response very slowly. Therefore, link availability analysis oriented on Web search has a great practical importance.By analyzing the influence factors of link availability, an analysis and speculating method for link availability based on the logic-level topological gragh of search results will be put forward. Besides that, the core conceptions in the method will be explained and definded, based on which the whole process of the method will be descripted and the core algorithms will be designed. The logic-level topological graph will be constructed with tree’s level-constructing idea by converting it to a binary tree. Stratified sampling algorithm considering search results’logical addesses will be put forward to get better speculating results. The algorithm of testing link availability based on HTTP request and ICMP messages will be put forward to obtain not only server information but also net information, which will lead to better speculating results. Rules for judging nodes’availability in logic-level topological gragh will be set up by analysing HTTP and ICMP messages, and the regulations for speculating link availability will be formulated according to the principle that the longer the testing path is the more available the link is. Finally, the speculating algorithm will be designed on the basis of judging rules and speculating regulations.All the core algorithms will be analyzed through experiments and the effectiveness of the method will be proved through comparison experiment.