Research On Adversarial Attack And Defense Methods For 3D Point Cloud Data

Making machines understand the three-dimensional world in which humans live is a fundamental problem in computer vision and an important prerequisite for downstream tasks such as virtual reality,augmented reality,and autonomous driving.With the improvement of computing power represented by GPU and the development of sensor technology represented by Li DAR and depth camera,the 3D perception method of 3D point cloud data based on deep learning has gradually become the most advanced method in the field of 3D computer vision.However,as 3D point cloud deep learning models are increasingly deployed in safety-critical real-world applications such as autonomous driving and robotics,the lack of robustness of 3D point cloud deep learning models has also received more and more attention.The research on the robustness of 3D point cloud deep learning model continues to advance with the game of adversarial attack and adversarial defense,and its ultimate goal is to develop a more robust 3D point cloud deep learning model.On the one hand,adversarial attacks on 3D point cloud data can discover flaws in deep learning models of 3D point clouds.On the other hand,the adversarial defense of 3D point cloud data can improve the robustness of the model and resist the harm caused by adversarial attacks.Although initial progress has been made in the adversarial attack and defense of 3D point cloud data,there are still some problems to be solved.First,the existing 3D point cloud adversarial attack methods generally perform poorly in the physical world and cannot effectively generate adversarial samples in the physical world.Second,existing 3D point cloud adversarial attack methods have low black-box transferability for generating adversarial samples,which limits the application scope of these attack methods.Third,the existing 3D point cloud adversarial defense methods will reduce the accuracy of clean samples,and at the same time,the characteristics of the 3D point cloud classification model are also ignored.Fourth,the insufficient robustness of existing 3D point cloud classification models to natural adversarial samples limits the real-world application of these models.Fifth,the current research on 3D point cloud adversarial training is relatively lacking.In order to solve the above-mentioned key problems,this paper constructs a 3D point cloud adversarial attack method and an adversarial defense method for different scenes,and builds a robust 3D perception system on this basis.The main innovations are summarized as follows:(1)Aiming at the shortcomings of the 3D point cloud deep learning model in different scenarios,two effective adversarial attack methods are proposed.One is the Mesh attack method.Through the differentiable sampling module from 3D mesh to 3D point cloud and multiple 3D mesh loss functions,the generation of smooth 3D mesh adversarial samples is realized,and the 3D point cloud adversarial samples are simplified.The reproducibility in the real physical world also greatly improved,and at the same time,the attack success rate of the physical world has been greatly improved.The other is the scaling and shear attack method that can be seamlessly combined with the existing 3D point cloud attack methods.It generates 3D point cloud adversarial samples for the scaled or sheared 3D point cloud,which effectively improves the success of the transfer-based black-box attack,achieving state-of-the-art transferability.(2)To improve the robustness of the 3D deep learning model in different scenarios,two novel 3D point cloud adversarial defense methods are proposed.Among them,the IT-Defense method of 3D point cloud makes full use of the permutation invariance of 3D point cloud to effectively improve the robustness of the model in gray-box scenarios without reducing the accuracy of clean samples.The Point Cut Mix method is a simple and effective 3D point cloud data enhancement method.It generates new training samples by mixing two 3D point clouds and their category labels,effectively improving the robustness of point cloud classification models under natural adversarial examples.(3)A very robust 3D perception system is constructed.Firstly,in order to improve the robustness of the 3D point cloud deep learning model under white-box attack,a local smoothing adversarial training method with adaptive attack distance is proposed.The attack distance is adaptively adjusted to effectively improve the defense performance of the adversarial training model under mainstream 3D point cloud attacks.Then,a robust 3D perception system is constructed by synthesizing the existing adversarial attack and adversarial defense methods in different scenarios.Finally,the unmanned vehicle experiments in the real physical world show that the system can defend against white-box attacks and also effectively defend against transfer-based black-box attacks,which greatly improves the robustness of the 3D perception model deployed in real unmanned vehicles.