The Design And Implenentation Of A Personalized Web Information Acquisition And Processing System

In recent years, with the rapid development of computer network technology and exponential growth of network information, it's becoming more and more difficult to find the information you need on the Internet, so the search engine was born. WebCrawler technology is an important technology of the search engine. The Web crawler technology and ETL technology, can form a personal network information acquisition and processing system.Urban management, including political, economic, social and municipal management, runs through the entire process of urban planning, command, construction, operation, safety supervision and coordination. Applying the personal network information acquisition and processing system to scientific decision-making in urban management will effectively improve the efficiency and accuracy.This article is based on a cooperation project with a research center, mainly completes the following work:(1) analyze the specific needs of this project, and analyze the feasibility and the advantages of the unity of the web crawler technology and the ETL technology.In this basis, propose the concept and the necessity of the personal web information acquisition and processing system;(2) according to the actual work of urban management characteristics and needs,use some of the more suitable ideas of the urban management to design a personal web information acquisition and processing system which is used for city managers; (3) use this design to implement a personalized web information acquisition and processing system.This system truly can be applied to public opinion analysis system,and test the performance of the whole system and verify the advantages of this system.