The Study Of Multi-Agent Cooperative Reinforcement Learning Methods

Reinforcement learning has been the hotpot in the research of multi-agent systems (MAS) and machine learning (ML), because it doesn' t require the environment model. In fact, a reinforcement-learning agent learns through its interaction with the environment. MAS is often applied into open, complex and dynamic enviroment, in which a single agent is insufficient to solve the faced task, so that agents must do their work cooperatively. In order to adapt to the environment' s dynamic changes, agents must have the learning capability as well. But the traditional single agent learning theory cannot hold true in the case of MAS. So it is desiderated to put forward a new learning method, according to the cooperative character of MAS.In Artificial Intelligence field, Pursuit game is often used to test learning algorithms, and for this problem, the thesis establishes two cooperative reinforcement learning methods for multi-agents: Goal Decomposing Based Learning (GDBL) method and Best Action Strategy Learning (BASL) method.As Game Theory (GT) reflects the social relationships among people or organizations, it is very appropriate to apply the GT to research the mutual relations in MAS. Based on this point, the thesis integrates Markov Games with reinforcement learning and makes preliminary explorations into the cooperative game reinforcement learning methods for multi-agents.After introducing some basic concepts of agent, MAS and multi-agents learning, the thesis presents the theory of reinforcement learning and some common reinforcement learning algorithms. On the basis of analyses of Pursuit game, the thesis establishes GDBL algorithm. However, the learning results of GDBL algorithm may be locally optimal, so certain improvement has lead to BASL. These two learning methods have been justified by experiments. Finally, based on the generalization and extending of these tow methods, the thesis makes some preliminary explorations of cooperative game reinforcement learning for multi-agents, and proposes relative algorithm with proof of its convergence.The main research achievements and innovations are the establishment of two cooperative reinforcement learning methods for Pursuit game, which are justified by experiments. Furthermore, Markov Games and reinforcement learning are merged into MAS with certain constraints so that preliminary explorations have been made to study the cooperative game reinforcement learning. On the basis of theoretical analysis, the cooperative game reinforcement learning method is proposed and its convergence is proved.