Implementation And Study On WEB Injection Vulnerability Detection Method

With the rapid development of Internet technology,network application platform has gradually become the main channel of information interaction,People's life service presents the characteristics of network and informationization.While the network brings convenience to people's life,it is also accompanied by a variety of data security problems.It has become more and more important to ensure the security of Web applications and timely discover and repair vulnerabilities.The essence of WEB Application vulnerability detection is to look at the security of WEB applications from the perspective of attackers.Based on the research and Analysis of Network Crawler technology and Vulnerability attack principle,the author improves the shortcomings of current vulnerability detection in order to improve the accuracy and efficiency of vulnerability detection.The main work of this paper is as follows:Aiming at the limitation of Web site to crawler,in order to solve the problem of complex verification code analysis encountered by reptiles in simulating login,this paper uses Cookie to simulate user login,by setting the relevant request header,bypassing the anti-crawler mechanism of the website,breakin,g through the restriction of the website to the crawler,and improving the crawl efficiency.Aiming at the problem that similar data exists in the URL crawled by the crawler,a HASH deduplication algorithm HDTSF based on tree structure feature is proposed,According to the structural characteristics of the URL,the URL data is grouped into subpaths with a depth of 2,and then the URL data within the same subpath is similarly deduplicated.At the same time,the HDTSF de-duplication algorithm,edit distance algorithm and cosine similarity algorithm are compared and tested.The results show that the HDTSF de-duplication algorithm can effectively improve the efficiency of the crawler.The WEB Injection Vulnerability detection program Icrawler-scan based on improved crawler technology is designed and implemented,which mainly includes URL crawl module,SQL Injection Vulnerability detection module,XSS vulnerability detection module and detection result report module.And the function test and performance test of Icrawler-scan are carried out,and the test results show that the Icrawler-scan program has good vulnerability detection effect.