The Research And Application Of A Vertical Search Engine In Internet Shopping System

With the rapid development of the information technology, Internet has become the largest data base in the present world, which provides the service of information share and communication. However, the rapid growth of the information brings us a large number of duplicate messages and spam appear on Internet, which makes finding available information become much more difficult than before, so how to get the useful information more quickly and more correctly is the problem which the Internet faces. In this situation, topic search engine which aim at special users and special themes comes into being, and also becomes a hotspot of the search engine. In this paper, our research revolves round this hotspot.In this article a shopping vertical search engine which applies in a shopping system is researched and designed. The paper describes the working theory and the main part of the general search engine, and the characteristics of the vertical search engine. To realize the system, the systematic frame of general search engine which is made of with web pages acquirement, web pages pretreatment and index, search and user retrieval is added. The paper describes the working theory and realization of these parts. The paper introduces the search strategy and the information filter of the topic net spider, how to pretreated the web pages to satisfy the index builder, how to build the index finally and how to sort the large amount of the search results. It also describes the two open source tool Heritrix and Lucene. During the process of the designing and the developing, much Htritrix and Lucene code is reused which strictly accorded to the software engineering request, changes the code to satisfy the system.The main differences between the shopping vertical search engine which is developed in this paper and the general search engine is whether it aims at the special users and the professional field. It is believed that the vertical search engine is available in the shopping system and is convenient and helpful to the users in this special field.