Design And Implementation Of Distributed Web Vulnerability Scanning System

With the advent of big data.era,the security of software has become a concerned topic.The reliability,practicality and security of software are the prerequisite for using the software.How to realize real-time data transmission,load balancing scheduling,real-time monitoring of system data and huge data storage are the problems need be solved.In order to realize web vulnerability scanning system based on distributed technology,In this paper,cloud computing is used for the data real-time tracking,distributed technology is used to develop a platform on the cloud.The vulnerability scanning system is developed with the MapReduce technology on the Hadoop platform.HBase technology is used to achieve the high store efficiency and rapid data processing which uses non-relational and column storage models to store data.The paper analyzed research present situation,then described simply related technology,and presented in detail various function modules.Hadoop platform in the development system includes a plurality of functional modules.First of all,different functional authorities of the different roles are set.According to the requirements of the system design access to modules(including different module interface design,etc.),management module(including management role,the scanner management,task management,etc.),monitoring module for real-time tracking and scanning data,statistical data analysis and so on.It is easy to analysis data and to make decisions for users,so that loopholes could be found and repaired timely.The module of storing data(by using HBase technology which stores data with column storage mode).At the same time,according to the load balance,different load balancing methods are designed and different task scheduling strategy are used,in this way the rational allocation of resources and the full use of server resources are achieved.The implementation of vulnerability scanning system could make developers discover loopholes timely and find the location of the holes.It could alarm and send the message promptly to notify the developers.Developers could locate the vulnerabilities in a timely and repair them.The system which developed could ensure the safe and reliable of the system and reduce or avoid the loss caused by vulnerabilities.