A Study Of Ajax-oriented Search Engine Techniques

The web as we know it is undergoing a significant change. A technology that has gained a prominent position lately, under the umbrella of Web2.0, is AJAX (Asynchronous JavaScript and XML), in which the combination of JavaScript and Document Object Model (DOM) manipulation, along with asynchronous server communication is used to achieve a high level of user interactivity.Mainly due to the fact that AJAX shatters the metaphor of a web’page’upon which many web technologies are based, this new change in developing web applications comes a whole set of new challenges, among which searchability and testability are the main concern.In this paper, we start our work from searchability and analyze the technical bottleneck traditional search engine faces after the emergence of Ajax. We do a full investigation of research status of search engine which supports Ajax and focus on the Ajax-oriented crawler technology.Since one Ajax page contains multiple states, we refer to a classic transition graph to model Ajax sites and introduce a single-thread Ajax crawling algorithm using it. On the base of the algorithm, we propose a parallel Ajax crawling algorithm and experiment results show that it brings performance improvement.Finally, we propose a prototype of Ajax-oriented search engine, and succeed in applying it based on a third-party search engine named Nutch, which proves the effectives of our points of view from a practical application perspective.