Research On Poisoning Attack And Backdoor Attack Based On Deep Reinforcement Learning

In recent years,Deep Reinforcement Learning(DRL)technology has been developing in various fields,bringing great benefits and convenience to society.By combining the perception ability of deep learning and the decision-making ability of reinforcement learning,DRL technology can directly map the original observation results to the operation behavior of the agent,and make the learning and decision-making ability of the agent more automatic and intelligent through end-to-end learning.Although the DRL model has good performance,many research works have shown that there are data security risks in both the training phase and the application phase of the DRL model.This is mainly due to the large amount of computing resources required to train such models,so more and more users tend to train models directly on the cloud platform.However,this practice provides an excellent opportunity for attackers to replace or mix the data set of the cloud platform with carefully crafted toxic data.Therefore,in order to make the DRL model more robust,it is a very meaningful work to study the attack and defense methods of deep reinforcement learning on data security.This paper mainly focuses on the research on the data security of deep reinforcement learning.Firstly,the existing data poisoning and backdoor attack methods are summarized.On this basis,a new research method is designed for the training stage and application stage of the deep reinforcement learning model,and the effectiveness of this method is verified through experiments.Specifically,the research content of this paper includes the following two aspects:(1)Data poisoning can contaminate the training data of a machine learning model,and tampering with the training data can affect the model’s ability to make correct predictions.Therefore,this paper proposes a poisoning method for the state sequence,which poisons the DRL model by replacing the adjacent interval states at critical times.And on this basis,this paper proposes a feasible poisoning data cleaning scheme,which mainly uses the target network of the DQN model as the verification network of the poisoning data to verify the poisoning data.Through this scheme,the influence of poisoned data on model training can be eliminated,which further complements the research on poisoning attacks in DRL.(2)A major limitation of current backdoor attacks is that the triggers are usually visible and easily identifiable under human visual inspection.Therefore,in order to solve the problem of insufficient concealment of triggers in backdoor attacks,this paper proposes a covert backdoor attack scheme for deep reinforcement learning.This scheme mainly uses ATN network to obtain important edge information of the image,and uses it as the covert trigger of this paper.The proposed method introduces hidden backdoor triggers that are invisible to human eyes in the DRL domain for the first time,making backdoor attacks difficult to detect by humans while ensuring that neural networks can still identify backdoor triggers.