Research And Implementation On Trading Information Oriented Vertical Search Engine

Vertical search engines are relatively general search engine large amount of information, inquires the inaccurate, such as the depth, not out a new search engine service mode, it through for a particular field, a particular people or a particular needs to provide a certain value of information and related services. Its characteristic is "designed, fine and deep", most industry colour.This thesis research general search engine and vertical search engine principle and key technology of the similarities and differences, and special focus on the vertical search engine professional web crawler and web structured information extraction technology research. Heritrix Lucene and is completely by the Java development, and open source software, they are respectively, provides a web spiders framework and full text search system framework, can facilitate rapid help us for a full text search system construction. Therefore, through the Heritrix Lucene and construct a full text search system plan, there is no doubt that the small and medium-sized enterprise full text search services provide a self rapid customization retrieval system shortcuts. This paper, by using Heritrix Lucene and designed a trading information for the vertical search engine system in information collection, mainly from fixed when web site for the computer network of trading information.Based on the design and implementation, vertical search engine for the web crawler technology of information collection module design, Heritrix to the function expansion, puts forward the other a more simple system function expansion mode: through inheritance Extractor and FrontierScheduler module, and its functions were perfect, meet the controllability of web grab.