Design And Implementation Of News Propaganda Information System Based On Data Crawling

With the rapid development of computer information technology and the explosive growth of network information,the data produced every day are numerous and varied.News data is information with strong timeliness.How to collect valuable news information from the massive information released by news websites has become an urgent problem to be solved.In view of the above problems,based on Python environment,this paper takes specific keywords as crawling objects,and on the basis of studying and analyzing the principles,core modules and running processes of current crawler technology,exploratively realizes multiple web crawlers for various websites,and completes data crawling and other goals.The purpose is to improve the efficiency of data mining in news management system,and to facilitate more scientific and systematic standardization of news management.The main contents of this paper are as follows:(1)Based on the implementation principle,workflow,crawling strategy,webpage text extraction and other related methods and technologies of common crawlers,the design of Python crawlers and the flow of data analysis are studied and analyzed.(2)This system is built by using Flask background program framework based on Python language,Layui front-end UI framework,My SQL database,and realizes a news management system which can automatically analyze news media websites,has high efficiency,and supports multiple crawler strategies.(3)After the system test,the system can log in,view the task information,crawl the data according to the key words,news management and task management.Through the system of efficient and rapid access to the news website valuable information,improve the news information system management capabilities and data analysis capabilities,greatly improve the efficiency of work.