Research And Implementation Of Vertical Search Engine

With the rapid development of the Internet and social progress, the amount of web pages becomes a huge number. The China Internet Network Information Center released a statistics on 2008, it said that the number will exceed to 10 billion. The network has greatly enriched the collection of information because the page number has a substantial increasing; on the other hand, it makes the traditional search engine much more difficult to get accurate results, and decrease the general user satisfaction. This is a chance for vertical search engine, while a traditional search engine can not provide accurate results without relative domain or other search rules.The vertical search engine is a new search service, aimed to solve the problems of too-massive data, low search accuracy and profundity in common search engine.This paper investigated a great deal of domestic and international documents, and deeply researched as well as the critical technique of vertical search engine. The paper focus on the subjects as below:(1) Describes the generic Web crawler and search engine based on its general structure and work processes; (2) Introduces the theme and a variety of webs crawling technology and sorting algorithms; (3) Do researches of Lucene full-text search framework, and accordingly, its improvements to meet the needs of vertical search engines; (4) Optimized the procedure used in reptiles of the vertical search engine, and implement a vertical search engine based on this.Experimental results show that the system is efficiency, has better collection of the pages related to the topic, has achieved the anticipated target. The system has good practical value and application prospect.