Research And Implementation Of Vertical Search Engine Key Technology

Today's society has entered the information age network, computer and network allows rapid development of information technology in all areas of data and information increased dramatically, and because of human involvement in the data and the uncertainty of information systems is more significant. How to mine information of value from large and strong chaotic interference potential of data,which gives information processing capacity of human intelligence presented unprecedented challenges. In some fields, such as real estate, e-commerce, the traditional search engines have been unable to meet the needs of users. Faced with these challenges, specific themes and personalized information retrieval vertical search engine came into being. Topic-based vertical search engine has become the search engine and Web information mining in a hot and difficult research, this paper is to study the hot and difficult technical and proceed.First, an integral part of the general search engines made a brief introduction, and a story of how it works. And then on key vertical search engine web crawler technology, such as subject, information extraction, text classification, and other vertical search engine architecture has been described in detail.Then, in the vertical search engine technology to explore the web crawlers which search strategy to access the Web, in order to improve efficiency and accuracy. Considering the theme of web pages exist on islands issue, topic-based analysis of the content and the URL link address of the search algorithm, so that web crawlers can crawl through tunnels more relevant to address the theme topic page silos, improve search engine theme resource coverage, and can better avoid the topic drift.Finally, the proposed method, with the design and implementation of a "higher education" relevant to the subject of vertical search engines, the main features include web crawling, web analysis, web page relevance judgments, crawl depth control, log and the results recorded, visual interface.