Multi-agent Reinforcement Learning Research And Application

For the existing of group agents, in MAS the influence of different agents can't be omitted, the circumstance is dynamic and complex; the learning agent in MAS must face such questions: how to use partial information for learning? How to achieve interactive learning with other agents? How to improve the learning efficiency? etc. In fact, none of the existing learning methods is sufficient for these questions; for solving these questions compositive learning and Multi-layer learning architecture are applied in this paper.This paper firstly proposes statistic learning based Q-learning algorithm for MAS, the agent can learn other agents' action policies through observing and counting the joint action, a concise but useful hypothesis is adopted to denote the optimal policies of other agents, the full joint probability of policies distribution guarantees the learning agent to choose optimal action. The algorithm can improve the learning speed because it cut conventional Q-learning space from exponential one to linear one. The convergence of the algorithm is proved.This paper proposes a learning framework for MAS; the framework consists of two levels. The high-level is a planner which is comprised of abstract control policies based on prior knowledge; the low-level is a predicting based Q-learning module. In learning, the prediction of next state will help greatly reducing the action search space. Planning is applied to solve the partly observing question. The learning efficiency of the framework is exceeding the conventional Q-learning.For the application in proposed reinforcement learning, neural network disturbing learning is discussed. Through introduce random disturbing into training process the algorithm can avoid plunging into local optimal minima, the random disturbing obey Boltzmann distribution, this guarantee the convergence of algorithm.This paper also demonstrates the application of algorithms and framework, the successful application in RoboCup illustrates the learning efficiency and generalization ability of the proposed algorithms and framework.