Research On Flocking Cooperative Control Algorithm Based On Reinforcement Learning

Group cooperative task is a common behavior in the animal kingdom.This kind of cooperative group behavior greatly improves the task efficiency of animals in prey search,predation and enemy control.Human beings also have tasks or needs similar to the collaborative work of animal groups,and expect to be realized in an intelligent way.Therefore,the bionic research on animal group behavior is of great guiding significance for the research of human group intelligence.Flocking cooperative control algorithm is a typical group control algorithm based on animal group behavior bionics.At present,There has been a lot of research on group control based on Flocking cooperative control algorithm,but the bionic research on the interaction between groups is quite weak.As an agent autonomous learning algorithm,reinforcement learning has potential value in improving the intelligence level of artificial groups(such as multi-agent system)and the efficiency of group tasks.Due to the intelligence of the reinforcement learning algorithm,artificial swarm intelligence is closer to biological swarm intelligence.Therefore,the main research goal of this article is aimed at the typical predatory behavior in the animal kingdom: wolves prey on sheep,based on the reinforcement learning and the flocking cooperative control algorithm,In this paper,the cooperative search of wolves in the process of predation and the process of confrontation between wolves and sheep are studied,and based on the bionic model,a multi-agent cooperative search system and a multi group multi-agent confrontation system are established.In the research process of multi-agent cooperative search,this paper mainly studies the non optimal search problem of free area multi-agent search algorithm,and proposes a distributed multi-agent area cooperative search algorithm based on reinforcement learning.In order to realize the application of reinforcement learning in multi-agent cooperative search system,this paper designs a ?-information map.The ?-information map can transform the continuous region searching process into the discrete ?-point traversal process,while ensuring no the dead area searching.When the communication of the agent covers the whole target area,agents can obtain the global optimal searching strategy by learning the hole search process.In the case that communication does not cover the whole target area,agents can obtain a local optimal searching strategy.In addition,based on the search algorithm proposed in this paper,the agent can obtain the global optimal search path by off-line planning,and realize the optimal search according to the planned path.Simulation results demonstrate that the required time for area search with the proposed algorithm is close to the optimal value,and the performance of the proposed algorithm is significantly better than the multi-agent cooperative search algorithm based on Anti-flocking.In the multi-group multi-agent confrontation system,through the redesign of the potential force of the flocking cooperative control algorithm,the simulation control of the movement in the process of the confrontation between wolves and sheep can be realized.In addition,based on the flocking cooperative control algorithm,a avoidance algorithm for sheep is designed to achieve sheep avoid predators while maintain the integrity of the sheep group.In this paper,a relative polar coordinate is designed to realize the discretization of continuous confrontation environment.In this kind of discrete environment,the application of reinforcement learning in group confrontation is realized,and a continuous discrete hybrid multi-group multi-agent system is constructed.Based on the distributed reinforcement learning algorithm,the autonomous decision of wolves is realized.The simulation results show the feasibility of the system.In order to realize the effective sharing of learning experience in the process of search and confrontation among the wolf group,accelerate the convergence speed of learning algorithm,and reduce the communication traffic of traditional cooperative learning algorithm caused by experience sharing,this paper proposes a distributed cooperative reinforcement learning algorithm with variable weight.And the convergence of the proposed distributed cooperative Q-learning algorithm with variable weigh is proved in theory,and the stability of the algorithm is proved by simulation experiments in the final.