Research And Implementation Of Scientific Research Information Management System Based On The Topic Web Crawler

With the development of the application of information management technology, the implementation technology of scientific research management information system is becoming more and more mature. Information of these existing systems is stored in database system, and the validity of information is not verified in general. So the information may be inaccurate in the later downstream analysis which brings a lot of inconvenience to the management of the scientific research information.In this dissertation, the system is based on the topic web crawler technology to realize the searching of scientific research information resources and the fetching of online information. After that, research information is classified according to user's requirement, and then stored in local server to achieve the validation of scientific information and some other daily operation. Through the retrieval of downloaded resources, duplicate downloads can be effectively avoided. It can provide powerful data management support for information management and information verification. The system is realized under the practical application background, using software engineering principle, software development method and ASP.Net technology to build a management system based on B/S (Browser/Server) mode. The crucial of this dissertation is to realize the key technology in the scientific research information management system. We have proposed a model of scientific information management system for the topic web crawler, and designed a set of relatively complete and feasible solutions. At the same time, the key technology of the topic web crawler realization is conducted.The web crawler and the scientific research information management are studied in this dissertation. The purpose is to establish the scientific research information management system based on topic web crawler. The main content of this dissertation is summarized as follows:1 In the background that the existing research information management system usually did not considered information verification. This dissertation introduces topic web crawler to the research information management system. A detailed system design solution is given based on the analysis of the functional requirements. Furthermore, the function and implementation method of topic web crawler is also discussed with respect to the information retrieving, downloading, saving and other issues.2 To realize the topic web crawler, this dissertation first studied the architecture and working principle of the traditional web crawler. And then, further research of the implementation of web crawler is conducted, including page parsing, the extraction of web content, etc. After that, according to the specific requirements of scientific research information management, the vector space model is chosen as the benchmark model for web crawler. Finally, the web crawler search strategy is designed.3 On the basis of all the research and design mentioned above, the scientific research information management system is implemented based on topic web crawler. Because of the introducing of topic web crawler, besides data capture, the system can analysis the dynamic interaction node simultaneously. And then, through a validation process, only authenticated the topic related information were stored in local server which could achieve the function of scientific research information validation.