Design And Implementation Of Web Security Scanning Tool Based On Web Crawler

With the rapid development of Internet,Web application systems have been widely used in various fields such as medical,education,financial management and others.Web application has the characteristics of usability,openness and developability those make its security environment terrible.Attackers use the vulnerabilities of Web applications to carry out malicious attacks,which not only seriously threaten users' information,but also bring huge economic losses to users and enterprises.The Web security problems have risen to national level,so that the country and the people have to pay real attention to them.In order to ensure the security of Web application,it is very important to identify vulnerabilities and fix them in time.Web application vulnerability scanning technology deminds to examine the Web application security environment from attackers' position.Firstly,the tool constructs a packet that detects a vulnerability and sends an HTTP request to the server.Then it judges whether the target site has a particular vulnerability and finds the security problems of the system by analyzing the response information.It can improve the security of the network system and protect the users' data.Based on the understanding of Web crawler technology,SQL injection and XSS,the author is able to master SQL injection and XSS attack mode through a large number of attack experiments and design a comprehensive test sample library.In this paper,the tool uses breadth-first crawl strategy to crawl Web pages and then distributes the crawling results to SQL injection detection middleware and XSS detection middleware.For possible injection points,the tool constructs targeted attack tests based on a large number of test samples.The results of the successful attacks are stored to the vulnerability database.Finally,the tool generates a scanning report.The specific work of the paper is as follows:1.I form the thought of the paper and organize the structure of the paper by learning and summarizing relevant research and design of Web security scanning tools.2.I study key techniques for vulnerability detection and complete the architecture design of the tool by doing research on Web crawlers,SQL injection and XSS vulnerabilities.Then by doing the research on Web crawlers,SQL injection detection tools and XSS vulnerability detection tools,I complete the detailed design of each function module of the tool.3.Based on the tools' architecture design and each functional modules'detailed design,I implement the functions of each module of the tool.4.Finally,I test the tool.I analyse the tools' scanning efficiency and accuracy according to the scanning results.I improve the function of the system,so that a better Web security scanning tool based on Web crawler is developed.