Research On Web Application Vulnerability Mining Based On Spider Technology

With the rapid development of Internet, more and more applications enrich our lives, such as online shopping and social networking sites. However, a great security risk also appears. Malicious users such as some hackers exploit the defects in Web application itself for obtaining information, stealing sensitive data or sabotaging the Web application in order to reap tremendous benefits. Therefore more and more attention are paid to Web application security.Web application vulnerabilities have different kinds so far. The one main reason that these vulnerabilities exist is the programming of Web application has security issues, the other one is the wrong management and permission control. Therefore, it is urgent to find an initiative method to find vulnerabilities and fix found vulnerabilities for enhancing the security of Web applications. However, there’s nearly no systems specifically detecting Web application security vulnerabilities and the existed systems function not well in domestic. So it is meaningful to study on the web application vulnerability mining.In this paper, we analyzed why web application vulnerabilities exist and introduce the types of web application vulnerabilities and related principles of web vulnerability detecting technology. And we design a system aimed at mining web application vulnerabilities based on the principles and explain the key technology. The main achievements are:1. We have done much research on the Web crawling and finally selected Larbin, which is an open source project with high quality on crawling web pages, to be the core of the web crawling module. Then we analyzed the defects of Larbin and enhanced its functions such as adding support on crawling pages on Chinese URL and develop the algorithm of judging whether page is crawled repeatedly. And the development declines the rate of false positive on judging if page is repeated significantly and increases the coverage.2. We also strengthen the identification of database information against SQL. injection, which has been the most common threat. The result is we develop the efficiency of simulated SQL injection attack and speed up the detection against SQL injection vulnerability. Through the SQL research on SQL injection deeply, we give out enough and sophisticated testing sets to ensure the effectiveness of detecting SQL injection vulnerability.