Based On Web Crawler Web Application Vulnerability Scanners

Along with the rapid development of network and communication, it's a trend to communicate with open network. The application of web becomes more and more prevalent. The website build and web page design become more popular, but we are bothered by the issue of network security problems. It has been attended at large. So the scanner has been studied and designed, the key technology has been researched and implemented in terms of the characteristic of the vulnerabilities in the web application programs. This makes the loophole scanning be exact and efficient.For a Web application vulnerability scanner, an excellent scanning design proposal plays a key role in the design of key technology, test, verification and implementation. And it has a direct effect on the final success or failure of the entire system. Combined with the characteristics of Web application vulnerabilities, the focus of this paper is designing a special and practical scanning system. By analyzing the key technology for vulnerability scanning, a Web-based Web application vulnerability scanning model and a set of relatively complete and feasible design proposal are proposed. At the same time, the key technologies and issues are studied in detail.In this thesis, vulnerability scanning technology is studied. And on this basis, a Crawler-based Web application vulnerability scanning system is designed. The main works of this paper are as follows:1. Many kinds of web application program vulnerabilities are investigated in detail, such as SQL injection and XSS vulnerability, including their cause, harm, attack steps, the way to avoid, detection methods, etc.2. Based on the open-source package libcurl , a HTTP client is developed and simulated, by which the exceptional requests and receive attack response are sended.3.Thread pool has been used to improve system efficiency in this paper, and tasks of the background modules are completed by the thread pool. The various modules are designed with the same architecture and class hierarchy, in order to improve the system scalability.4.The web crawler technology is investigated in detail. Using web application program vulnerabilities characteristics, the function of web crawler has been improved. Dynamic interactive nodes are analyzed in the process of crawling, in order to record possible vulnerable points.5.The technology, work structure, basic framework and the shortcomings of existing Web-based Web application vulnerability scanners are studied in detail. A Web-based Web application vulnerability scanner is designed and implemented, and its system structure and main function modules are descripted. Based on system requirement, the relevant implementation technology of each module is studied.6.The scanner was tested, and the test result was analyzed. And the results show that the scanner is effective.