Web Vulnerability Mining System

Abstract: With the rapid development of the Internet in the WEB2.0era, thebarriers to web site development are getting ever lower, and also increasing thenumber of websites. However, there are many Web application vulnerabilities mostsites. According to statistics,75%of destruction occurs in the Web-client. Forinstance, Web Trojan, invasion of website due to SQL injection attack,modificationand lose of significant data and so on.HTML5has become more and more popular,but a large quantity of vulnerabilities are potential because of the lack of vulnerabilitydetection.Bases on Web application development under the status quo of low-threshold, thispaper develops a system, which could detect vulnerability in all kinds of webpage forHTML5to meet the detecting requirement of Web developers. The system adopts webcrawler, Multithreaded task scheduling and many other technologies, realizingvulnerability exploiting caused by SQL injection attack,detection of malicious linkand XSS, Web Trojan detecting, etc. key techniques to make vulnerability detectionmuch more accurate and efficient.The main work of the system has completed as follows:(1)Use domain name recursive query technology to achieve the sub domain nameinquiry of target site, and then realize a efficient Web crawling algorithm bases onBreath First algorithm and Bloom-Filter algorithm, providing basis for subsequentwebpage vulnerability detection.(2)To against malicious links and malicious script code, this paper uses a reversedomain lookup technology and algorithm based on entropy of information to achieveeffective detection and identification.(3)System combines stochastic fuzzy testing and injection testing method togetherto analyze various attributes of webpage efficiently, achieving the detection of staticand dynamic web pages. Injection tests are mainly based on the analysis of existingsecurity vulnerabilities, using penetration testing method to achieve vulnerabilitydetection; fuzzy random testing in accordance with the composition of the HTTPprotocol, and achieve vulnerability detection mining by structuring random structureof testing for different field. Finally, after testing for detection module and the integrated system, the system canmeet all kinds of known vulnerability detection and has a significant effect forenhancing the security level of the target site.