A Multi-agent Reinforcement Learning Algorithm Based On Sparse Interactions

Nowadays,with more and more life scenes changing from static single agent to dynamic multi-agents,Multi-agent Reinforcement Learning methods are receiving more and more attention and application.However,the problems of curse of dimensionality and huge communication costs between agents faced by multi-agent reinforcement learning are becoming more and more serious.Based on the introduction of the existing sparse interaction mechanism,this paper establishes a new optimized framework of sparse interaction,and proposes a multi-agent reinforcement learning method based on efficient coordination.Good performance of simulation is get in multi-agent grid tasks.The multi-agent reinforcement learning method with sparse interactions based on efficient coordination proposed in this paper mainly includes two parts.The first part is to effectively combine the single agent learning process(Markov Decision Process)and the multi-agent joint learning process(Markov Game)to establish a new sparse interaction framework,which separates the Q-value updating rule in joint states from non-joint states.The second part is aimed at the collision coordination problems between agents,and pure strategy equilibrium is adopted to get the optimal joint action strategy.Based on pure strategy nash equilibrium and nonstrict equilibrium dominating strategy profile,chicken game model is introduced and a method for solving chicken game equilibrium is proposed.In addition,the efficiency of the designed algorithm is verified by MATLAB simulation in multiple sets of grid tasks,which can help multi-agents learn(close to)optimal action strategies in path planning problems with lower storage space,lower computational cost and less learning time.