Research On Soccer Robots Cooperation Mechanism Based On Markov Decision Theory

The coordination and cooperation mechanism of multi-agent systems (MAS) is one of the most popular research areas of AI, which is of much practical significance due to the widely usage of MAS.In this thesis, the coordination and cooperation mechanism of agents is studied based on Markov decision process (MDP). The main results of this work are as follows:First, we proposed an action selection algorithm for well-communicated central control situations, which is based on MDP hierarchical decomposition method, by combining the concepts and methods of game theory and predicting the reword value of player’s strategies. The algorithm was tested in the central controlled FIRA2D robot soccer platform, which gave rather good performance in the games.Next, for the problems of continuous state space and large scales of distributed controlled MAS with bounded perception and communication, a real-time policy planning algorithm called MAXQ-RTP was proposed, which incorporates the MDP task decomposition method based on the MAXQ value function hierarchical decomposition. The algorithm adopts a framework of fully using the perception and communication information, employing AND-OR graph to describe the possible policies of each agent, and performing real time optimal planning.The main experimental work was performed on the distributed controlled RoboCup2D platform. We used MAXQ hierarchical decomposition to model the soccer agents, applied the MAXQ-RTP algorithm to plan the optimal strategy in real time. Experimental results showed good performance of the MAXQ-RTP algorithm.Due to the simplification of other soccer agents’reactions in the MDP model, some good strategies may be lost. In the following research we will extend the MAXQ-RTP algorithm in the game theory framework, taking other agents’reactions into consideration to make better decisions.