Research On Key Technologies Of Web Application Security Detection

Though the WEB technology has been developing over just20years, it has become one of the most popular application architectures. The security issues of WEB application system have also been the key points of information security and network security concerns. With the rapid development of WEB technology in the worldwide, the traditional WEB application security detection technology can not satisfy the current detection requirements, which can be concluded as:the existing detection technologies adopt the method of complete scanning system with so many redundancy and the harvest rate and efficiency cannot adapt the increasing scale of the WEB application; The traditional detection technologies have poor abilities to interact with WEB application automatically, and cannot interact through the complex, multi-type interfaces effectively, which causes high false negative of security detection; Existing researches on detection technologies still focus on results driven method mostly and ignore the detriments to the detected objects, leading to its limitations on detecting current WEB applications; Moreover, with the rapid development of WEB technology and the diversity of security thread, existing detection tools or platforms show some shortages in many aspects, such as in expansibility.The target of the research is to solve these problems above through the follow4aspects:Firstly, for the problem of low harvest rate and detection efficiency, WEB information crawling model is proposed based on reinforcement learning. The basic idea of the model is classifying vast amounts of WEB information according to their interface relevance and then crawling WEB information selectively in accordance with the special needs of detection, targeting the information crawling process. In this model, considering the characteristics of the relationship between interfaces and links, a "combined rewards evaluation algorithm" is designed based on Q-Learning, in which immediate rewards combined with future rewards after weighted. Immediate reward and future reward are assigned with different weight in different period (Learning period/exploiting period). While detecting, WEB information crawler based on Q-learning is trained firstly and the Q-value table is established. Then the integrated Q-value is calculated according to the learning results and the link is selective required according to the integrated Q-value. At the same time with crawling, the learning rules will be optimized. Additionally, in order to make it convenient for WEB information extracting and analyzing, a web page structuring method is presented based on cluster analysis. This method analysis the tags information, content information, area information and others in web page, and generates a structured tag tree (TAG_TREE) for each web page. Then through the method of cluster analysis, the TAG_TREE will be compressed and simplified according to the area information. Finally, a compressed TAG_TREE is generated, which makes preparation for extraction of WEB page characteristics.Secondly, in order to solve the problem of auto interact through complex and variety interfaces in WEB application security detection, a method of auto interact with web application is issued based on information feedback. In this method, all types of public interfaces are modeled according to their characteristics, and are described with a unified formal expression. Meantime, a value set is built to store the interface assigning data. In the process of interaction, the structure of value set is adjusted according to the analyzed results of feedback information, which could improve the accuracy of value assignment. According to the features of interfaces, the logic units in each interface are divided into two types, finite set logic units and infinite set logic units. The former is assigned with default values of itself, and D-BPM-BM algorithm is used to assign correct value to the latter, which could make the infinite set finite through fuzzy matching characteristics and selecting the most feature similar value set. The interactive method presented would improve the success rate of automatic interaction significantly.Thirdly, a web security vulnerability detection strategy based on attacking simulation is proposed, in which a policy of "Request Rules Selection" is adopted. The detection strategy improves the problem of "testing detrimental" existing in the result driven detection method. The policy of "Request Rules Selection" has three request rules, complete rule, common rule and safe rule. According to specified requirements, suitable request rule could be applied. Based on the detection strategy proposed, detection methods for XSS vulnerability and SQL injection vulnerability are also studied, and bypassing technology is adopted in the former and strategy of vulnerability confirming based on multi process analysis is used in latter.Finally, with the purpose of adapting to the rapid development of WEB technologies and the variety of WEB application security thread, WEB application security detection platform architecture based on "three-lays&two sides" is designed. In the architecture, detection, analysis and management are distinguished strictly, and broad interfaces ensure data exchange among modules. The platform architecture greatly improves the expansibility of detection platform.The feasibility of the models and methods is proofed through many experiments. And the improvement of effectiveness of WEB application security detection is verified by practical experiments.