Multi-Agent Systems Leader-Following Pursuit Based On Reinforcement Learning

In recent years,with the development of artificial intelligence and industrial technology,many control scenarios with large scale and complex structure have appeared.Faced with these complex control scenarios,the problem of limited ability of the single agent has become increasingly prominent.The emergence of multi-agent systems has significantly improved the intelligence of the control systems and overcome the shortcomings of the single agent.In practical engineering applications,the multi-agent systems pursuit problem shows great application prospects,which is involved in military confrontation,post disaster search and rescue,unmanned driving,civil security and other fields.Therefore,the research on the multiagent systems pursuit problem has very important engineering application value.At present,researchers mostly use the methods of virtual physics,game theory,formation control and other methods to solve the multi-agent systems pursuit problem,but most of these methods require mathematical model of the environment,which is difficult to solve the multiagent systems pursuit problem in engineering applications.This paper mainly investigates the multi-agent systems leader-following pursuit problem with unknown environment model by reinforcement learning.The main contents of this paper are as follows:(1)Aiming at the multi-agent systems pursuit problem with single evader in a simple environment,and only some pursuers can detect the evader.The method which combines the leader-following control and Q-Learning is proposed.The pursuit problem is solved by assigning tasks to the pursuers and designing reward function.(2)The multi-agent systems pursuit problem with single evader in a complex environment is studied.And only some pursuers can detect the evader in the problem.The method which combines leader-following control and hierarchical Q(λ)-Learning is proposed.Q(λ)-Learning is utilized to all pursuers.The pursuit problem is solved and the pursuit efficiency is improved by introducing hierarchical Q(λ)-Learning to the leader pursuers and improving the reward function.(3)This chapter investigates the multi-agent systems pursuit problem with multi-evader in a complex environment,and only some pursuers can detect the evaders.The method which combines leader-following control and experience replay Q(λ)-Learning is proposed.The pursuit problem is solved and the pursuit efficiency is improved by introducing experience replay,redesigning the individual task assignment strategy for the pursuers and proposing the dynamic task assignment method.