Research On And Realization Of A Vertical Search Engine

With the volatile increase of Internet information, both information overcast rate and searching precision of universal search engine fall continually. But the demands on getting more correct and detailed information increase continually. Vertical search engines appear in this situation.A vertical search engine searches professional information in a field or industry and it is extension of search engine. The vertical search engine has some characteristics compared with the universal search engine, it can obtain more accurate and professional information.This paper studies the theory and application of the vertical search engine. A prototype vertical search engine system is designed and realized based on a URL selection method proposed in this paper, which combines content-based analysis and link-based analysis.Firstly, the paper analyses some problems in the universal search engine and shows that the research of vertical search engine is necessary. The structure of vertical search engine is discussed. Information collection and index part of the vertical search engine are deeply analyzed.Secondly, the paper researches on URL selection technology of the vertical search engine and analyzes it in detail. Based on the analysis, the paper proposes a DKWT URL selection method for vertical search engine. This method combines content-based analysis with link-based analysis. The DKWT method can get similarity degree between web content and focused information. This similarity degree is based on the vertical distance of web, some web content and web links, in addition, it can determine whether this web content can be crawled. The DKWT method can also predict the similarity degree between web content of sub-links and focused information.Finally, the paper realizes a prototype vertical search system based on the DKWT method. The simulation results show that, compared with content-based URL selection method and link-based URL selection method, the method proposed in this paper can obtain the focused information fast and effectively.