Research And Application Of Crawler System In Network Information Resources Based On SOA

Internet is the most important channel to transmit information as the largest information database in the world. However, the characteristics of information such as tremendous volume of data, a low degree of integration, and so on, restrict us to fully mining the value of the implication of information. How to gain the valuable information effectively has become the hot issue in computer industry. Fully studying on the ontology technology, the paper puts forward a Crawler Algorithm based on directory Tree in network information resources and the fetch Model by visualization rules to solve the problems on web-page link and text extraction. And with the technical program demand of the project of Resource Integrate in Science and Technology (RIST), service-oriented network information crawling system is designed and realized to integrate the network resources effectively. This system has proved the feasibility and practicality by application in RIST.