A Study Of The Trade-Off Between Robustness And Accuracy Of Deep Neural Networks

Recently,deep neural networks have grown rapidly and achieved great success in many tasks of artificial intelligence,but they have also proven to be highly vulnerable to adversarial attacks.As one of the most effective defense methods,adversarial training tends to compromise the accuracy of the model while improving its robustness.In fact,different application scenarios may place different emphasis on each,so the trade-off between the two deserves further exploration and research.In this thesis,we consider a more favorable trade-off and a more flexible defense algorithm based on the idea of adversarial training.The main tasks are as follows:1)In order to flexibly regulate the trade-off between robustness and accuracy,this thesis solves the internal maximization problem of adversarial training based on a new perturbation budget design and proposes group adaptive adversarial training.The method divides the training dataset into groups based on the example’s own antiperturbation ability,treats the groups as a whole,and determines the group perturbation budget for each group based on the binary search method.The search process consists of two stages,expansion and contraction,to ensure the reasonableness of the perturbation budgets.Experimental results on two classical image datasets,CIFAR-10 and Image Net-30,confirm the algorithm’s ability to adjust flexibly and to achieve a better RA-NA trade-off than classical defense methods under white-box and gray-box attacks.2)Considering that over-inclusive decision boundaries bring robustness while greatly hurting accuracy,this thesis proposes a moderate-margin adversarial training algorithm that aims to learn ideal decision boundaries.First and foremost,as the target of model training,the loss function is the key factor affecting the final performance of the model.The algorithm designs a hybrid loss function to learn adversarial and natural examples separately.The former are fine-grained adversarial examples,and the perturbation budget and iterative attack steps are set based on margin to mitigate severe crossover with their adjacent natural examples.The latter are those that are hard to correctly classify by themselves,and their decision boundary can be tuned to be close to the naturally trained model.Experiments on multiple datasets confirm the effectiveness of the algorithm,and comparisons with multiple defense methods further show that the algorithm achieves a better robust-natural accuracy trade-off.The research on adversarial training methods in this thesis has some theoretical and practical significance for the task of classifying natural and adversarial examples.On the one hand,this thesis explores the learning process of adversarial training and provides new ideas for the design of training strategies; on the other hand,the methods proposed in this thesis are transferable across multiple datasets and models and can be used for the deployment of robust models in other fields.