Research And Design Of Vertical Search Engine Based On Fish-search Algorithm

The rapid development of Internet leads to the rapid growth of network information resource, general search engine has been unable to meet the needs of users in the aspect of information retrieval, today, information retrieval is the pursuit of accuracy, at the same time information retrieval should have some depth. Though information which is returned to user is enough, it is not the information what user wants, so though the general search engine gives many results, its accuracy of results is a little worse. For this problem, the specialized vertical search engines have been put forward.Vertical search engine is a search engine for specific field or specific topic, has greatly improved the accuracy of the search results, to some extent to meet the needs of the user retrieval. The main research in this paper is the design of vertical search engine, the core module of vertical search engine is the focused crawler. The focused crawler is to crawl the webpage which is relevant to the topic, and download the webpage. So the key is how to judge the correlation between topic and webpage, this paper uses improved Boolean model to calculate the correlation which is between topic and webpage, correlation not only determines whether the current Webpage need to store, but also is the reference value to climbed link priority in the linked list, and it is the main basis of the search results ranking; in addition, the focused crawler crawling strategy is using the improved Fish-search algorithm. After downloading the webpage, it also needs to extract important content as a retrieval condition and webpage information description, the webpage information extraction method is a regular expression that matches the related information and extracts the correct information. While users retrieve required information, it can recommend some of words related to the searching word or phrase according to the input keyword, provides a more friendly interface for the users.The paper finally completed the design of vertical search engine, compared with the general search engine, when searching the specific topic keyword, vertical search engine which is designed in this paper is much more accurately than general search engine in retrieval results, and most of retrieval result pages which are searched by search engine which designed by this paper have good correlation with topic. So in a certain field, vertical search engine is designed in this paper has great improvement in the accuracy of the retrieval results, it basically achieves the purpose of the research in this paper.