Web Spider Design And Realization Of Intelligent Search Engine

The spider programming technology is the key part of search engine, which is the convenient and effective method to get the information from the WWW. Surrounding the innovative technology of intelligent search engine and based on the whole request of its frame, the main work of this article is to realize the cruise of the internet spider , and store the data of the page into the local database, place a firm foundation for the realization of intelligent search engine.The main content of this article is:Firstly, analyze the principle of the search engine, realize the first step in the work of search engine: get the page data of internet. Secondly, describes particularly the technology of Java , especially the technology used in this article , such as connection of Socket, connection of JDBC, data stream of Java, read and write of blob field, thread.Based on the network spider software, this article analyzes and designes a system of a new spider. Combined with multi-threads technology , this article realizes the supervisor of alignment based on SQL and page data of internet stored in local database.Thirdly, after analyzing the result of experiment in the network of campus and the result of the data stored in the database, the feasibility of the spider can be validated , the prospective object of the system have been achieved.At last , the conclusion of the whole system and the next content of this subject are presented.