Research On Application Of Reinforcement Leaning In Graph Adversarial Attack And Policy Transfer

Reinforcement learning(RL),which is designed for solving sequential decision-making problems,is an important machine learning technology.After a long period of development,it has been successfully applied in many fields e.g.robotics,recommendation systems,and automatic control.Though,reinforcement learning has achieved great success in these fields,it is extremly data inefficiency.Training an agent from scratch requires considerable numbers of interactions with the environment for a very specific task,which limits its application in some practical situations.One way to improve its data efficiency is transfer learning,which uses the knowledge learned from the source task to accelerate training on the target task.However,most of the existing transfer reinforcement learning studies attempt to learn a common feature space of states across related tasks to exploit knowledge of state spaces as much as possible,ignoring the similarity between action spaces.In this thesis,we attempt to apply reinforcement learing to the research of adversarial attacks on graphs.And then,propose a framework of policy transfer based on action embeddings to improve the data efficiency.The main work is summarized as followsFirstly,we introduce a method to perform adversarial attacks on graphs by adding fake nodes,and propose a reinforcement learning based algorithm.Firstly,we model the process of adversarial attacks on graphs by adding fake nodes as a Markov Decision Process.The state contains current graph structure and the attack target,the action is represented by the nodes in graph.Then we use a DQN based method to evaluate the effect of each action and select greedily.The experimental results on three public datasets show that the proposed method can effectively attack the graph convolutional networks and the adversarial examples generated based on one model can be transfered to others.Then,we propose a state transition model based method to learning action embeddings and introduce a framework of policy transfer based on action embedding.Action embeddings are learned from trajectories generated by any policies.And the policy transfer is achieved by passing on the parameters of policy and transition model from the source task.We evaluate our methods on two sets of simulated environments and a set of real game scenarios.The experimental results show that our methods can learn action embeddings that contain the senmantic of actions and effectively transfer policy across tasks with different state and action spaces.