Research Of Multi-agent Cooperation Mechanism Based On Reinforcement Learning

Multi-agent system is composed of multiple interacting autonomous agents, the core issue of the research is to seek to establish an effective cooperation mechanism, making the simple and independent agents complete tasks or solve problems of complex targets through consultation,coordination and cooperation.Reinforcement learning algorithm learns through trial and error with the dynamic environment, which is an important branch in the field of machine learning and artificial intelligence, and is a leap to control agents through rewards and punishments, instead of task implementation.This thesis focuses on multi-agent cooperation mechanism for large-scale, real-time, and uncertainty problems, based on the reinforcement learning algorithm. The main work and results are summarized as follows:1. Based on the Markov properties of the problem, several kinds of Markov decision process models were introduced. Considering the partially observable property of multi-agent problem, the partially observable Markov model was built, and converted to semi-Markov model to meet MAXQ-Q algorithm.2. In view of large scale features of the problem, MAXQ-Q hierarchical reinforcement learning algorithm was introduced to decompose the multi-agent problems whose state-action space is huge, and algorithm complexity is in exponential growth, into small linear semi-Markov decision problems which can be solved recursively.3. In view of the MAXQ reinforcement learning algorithm proposed in this thesis, the and-or graph was applied for strategy representation in large state-action space strategy search problems, to combine with the state abstraction methods for state reduction to meet the real-time demand in soccer robot problems, such as action selection and state transition.4. A new multi-agent cooperation framework was proposed based on the above work, and its real-time performance and stability were tested in Robocup2 D soccer robots simulation platform and a cooperation task with two NAO robots developed by Aldebaran Co., a France company.