Research On Application Of Reinforcement Learning In Multi-Robot Games System

Multi-Agent System (MAS) is often applied into open, complex and dynamic environment due to its characteristics of group, cooperation and intelligence. It can solve the complex issues which cannot be handled by sigle agent. Reinforcement learning (RL) is an important machine learning method. It makes the agent learns knowledge through continus interaction with the environment. Reinforcement learning has been the hotpot in the research of MAS and machine learning. Meanwhile, the traditional sigle-agent RL theory cannot hold true in the case of MAS, because the MAS RL faces more complex environmental changes and collaboration issues.The major problems of RL in Multi-Robot Games System include:(1) How to avoid the curse of dimensionality which is caused by the increased agent number and the dynamic change of environmental state; (2) How to allocate the reward signal after MAS obtaining the signal during the RL.This study concentrates on the research of reinforcement learning method in multi-robot agent games system, and proposes preliminary improved method for RL according to these two issues. Firstly, the multi-agent reinforcement learning method based on K-Means algorithm is researched in this paper. With this method, the environmental states were classified into different clusters with K-means algorithm, the combination explosion of state space was avoided according to the relationship between state clusters and strategy. Experimental result shows that this method will efficiently reduce the state space and improve learning speed. Secondly, a credit assignment method based on role clustering is presented, the roles can be dynamic assigned to agents and the corresponding set of characteristic behaviors is established by using K-Means algorithm. Thirdly, a new credit assignment function based role clustering is researched to solve the credit assignment problem in multi-agent systems, which is proposed according to factors like the weight of role and the implementation of actions. The experimental results show that, compared with the rival with credit average allocation strategy, the cooperation success rate, offensive and defensive capabilities of the team with credit assignment were significant improved.