Correlation Between Coverage And Adversarial Examples For Deep Neural Networks

In recent years,deep neural networks and deep learning have gradually entered the more and more people's lives.The widespread application of deep learning systems will bring convenience to human life while also bringing some hidden dangers.In some safety-critical applications involving personal safety and property safety,deep neural networks must be fully tested to eliminate hidden dangers as much as possible.Deep neural networks as the basis of deep learning systems should be fully tested.However,there is an essential difference between deep learning systems and traditional software testing,so traditional software testing techniques cannot be directly applied to deep neural network testing.Many scholars in related fields have proposed coverage criteria based on deep learning tests,but whether these criteria are useful is unknown.This paper studies coverage-based testing in deep neural networks,using indicators such as peak coverage,the number of test cases required to reach the peaks,and the time required to calculate the coverage to evaluate coverage criteria,and studies the feasibility of using coverage to guide the selection of test cases to pick out adversarial examples.The main research work of this paper is as follows:(1)Using MNIST and CIFAR10 datasets to train a total of six experimental models andadversarial data generated using four adversarial sample generation techniques.For eachcoverage criterion,the peaks of coverage,the number of test cases required to reach the peaksof coverage,the coverage of different layers in the network,the time required to calculate thecoverage,the statistical relationship between the coverage and the ability of the adversarialsample detection capabilities are studied separately.The results show that the correlationbetween coverage and the proportion of adversarial samples is weak.(2)Using MNIST and CIFAR10 datasets to train a total of six experimental models andadversarial data generated using four adversarial sample generation techniques.For eachcoverage criterion,use it calculating the coverage rate of the data sets,and select the test casesfrom the data sets based on the coverage rate.Under the same test case set,the ability to detectthe adversarial samples of each coverage criterion is statistically compared.The results showthat in the deep neural network,the effect of selecting the adversarial samples based on thecoverage is poor,and the detection ability of the adversarial samples cannot be improved.