The Design And Implementation Of Adaptive Security Scanning Task Scheduling System

With the popularity of the Internet,people's information is bundled with Internet applications.However,many applications have security vulnerabilities.There are already many security scanners available today,but the reports are not accurate enough.Some items can be modified,supplemented and excluded after later verification.However,the artificial verification by the expert security team is costly and easy to be affected by personal factors.That has always been the bottleneck of the implementation of vulnerability verification.Also current security platforms can only integrate single type scanners and cannot meet the requirements for scanning multi-type Internet applications.The scaning task scheduling system mentioned in this thesis and the crowdsourcing verification platform it belongs to are designed to solve the problem.The platform combines automated security scanning by machines with crowdsourced verification by the security team.Adaptive task scheduling system integrates various scanners to achieve unified platformization.The solution not only realizes the diversified integration of security scanning,but also reduces the cost of secondary verification and improve the accuracy of reports based on downstream crowdsourced verification.Due to various specifications and inconsistency of the scanning interface of the scanning tools,the system needs to be equipped with an adaptive task scheduling system to provide stable and reliable scanning reports for crowdsourced verification.The task scheduling system can adaptively satisfy task distribution and scheduling among different mechines.It can also process report collection,dirty data cleanup,unified format conversion and storage.The system is developed based on Java with SpringBoot framework and uses Mysql as the database.It provides unified format converter to interact with scanners.The task scheduling algorithm is optimized based on weight round robin scheduling and combines synchronous and asynchronous scheduling.