| With the rapid development of network technology, the Internet is generating large amounts of data everyday, the number of various software applications is on the rise, and the softwares update faster and faster. How to extract valuable data effectively from such a flood of information is an urgent thing to resolve.According to the actual needs of software update information requirement, the thesis researchs and develops the software update information system. The system includes web crawler and Web site development, main work includes:Firstly, it collects and analyzes various website of software information, determines the Web crawler implementation scheme, uses Python and best first search algorithm to realize web crawler, and the thesis studys multi-threaded web crawler algorithm, proposes a method of combining multi-threading and queue management to improve the collection efficiency, and avoid "crosstalk" problems caused by multi-threading;Secondly, to be user-friendly and intuitive for user to check the software information, uses Java to develop the website, research JFinal framework, bootstrap and other technology s to develop website quickly, and provides e-mail subscription function for users;Again, it uses MySQL database software to store information and e-mail subscription information;Finally, test the software update information system in practical applications, and the performance can meet the existing requirements of the company. Test data of the actual system indicates that the system has a certain optimization on collection efficiency and accuracy, and verifys the reliability and validity of the designed system. |
PDF Full Text Request