Collaborative Confrontation Algorithm Based On Deep Reinforcement Learning

Agent refers to a complete individual system that perceives and interacts with the environment,and can use existing information to iterate on its own strategies.Deep reinforcement learning technology is often used in agent modeling tasks due to its powerful feature extraction and decision-making capabilities.With the development of artificial intelligence and the deepening of research on the problem of agent modeling,the multi-agent system composed of multiple agents has made more accurate modeling of real-world application tasks due to its powerful expressive ability.The problem that has received wide attention from scholars is the problem of multi-agent collaborative confrontation based on reinforcement learning,which aims to study how multi-agent groups based on reinforcement learning can defeat other agent groups through optimal collaborative decision-making in a complex and changeable environment.Multi-agent collaborative confrontation algorithm technology based on reinforcement learning is widely used in practical tasks such as battlefield confrontation,game AI,robot control,etc.It has strong research significance and value in both theoretical and practical applications.The multi-agent collaborative confrontation environment constitutes a huge strategy space due to its complex environment and a large number of agents,and the lack of algorithmic expression power will lead to difficulties in convergence.And because there are unpredictable other agents in the multi-agent collaborative confrontation problem,how to construct a robust agent algorithm is also of great significance.This article reviews and summarizes the current research history and status quo in the field of reinforcement learning and multi-agent systems,combined with existing work for in-depth research,and puts forward the following innovations to solve the above problems and conducts experimental verification:(1)Aiming at the huge strategy space of the multi-agent collaborative confrontation environment,this paper proposes a multi-agent reinforcement learning algorithm based on strategy layering(SL-MARL).Through the idea of two-level strategy stratification,the upper-level macro strategy algorithm and the lower-level micro-action execution algorithm are constructed based on reinforcement learning technology.In the upper layer,considering the complexity of the multi-agent system and the interaction between the agents,a value decomposition network based on the attention mechanism and the sub-state space is constructed.Since the lower layer needs to accept the policy input from the upper layer,a value network based on the macro strategy is constructed.Experiments show that the algorithm is superior to the baseline algorithm in terms of convergence speed and performance in complex environments.(2)Aiming at the problem of insufficient robustness of agents,this paper proposes a collaborative confrontation algorithm evolution method(EBDP)based on diverse populations.Pretraining is carried out through different training objectives to construct a diverse and rich initial agent.On this basis,a mixed agent confrontation pool is constructed through different parameters,and the agents are sampled so that the agents conduct confrontation training with each other and continuously generate new agent parameters to join the confrontation pool,and improve the performance and robustness of the agent through continuous iterative training.Experiments show that after EBDP training,the performance and robustness of the agent can be improved.