Research And Design Of Soccer Robot Decision-making System Based On Multi Agent Reinforcement Learning

How to make the agent to think and make decisions like the human is theessence of artificial intelligence. To solve the markov decision process describeddecision making problems under uncertain environment, the agent use themethod of trial and error in the environment which called reinforcement learning.RoboCup (Robot World Cup) which is the international robot soccercompetitions and academic activities is aim at to promote the development ofmulti-agent system and distributed artificial intelligence in related areas. Amongall the leagues, Robocup2D simulation competition is a league focus onmulti-agent system decision making problems.The main manner of this paper is based on multi-agent reinforcementlearning, especially, MAXQ hierarchical reinforcement learning method andcollaborative action learning. This paper regards Robocup2D simulation game asthe experimental platform for multi-agent reinforcement learning and decisionmaking problems to do the research and apply the related methods andachievement on players’ privy skills and cooperation actions.Firstly, this article analysises and summerizes the theory of multi-agentsystems decision-making and reinforcement learning. Secondly, in order to sovlethe complex dimension disaster problem of the state of reinforcement learningsystem, this article applies CMAC neural network as the way of generalization toaccurate the speed of learning and improve the accuracy of the learning result.Using simulated annealing strategy, so that the learning process can escape fromthe local optimal solution and obtain the global optimal solution by exploration.In addition, the way of MAXQ hierarchical reinforcemnt learning has beenimplemented through a hierarchical structure of decision-making tasks, whichaccording to the system environment and the complexity of the task to be divided.In this manner, the large dimensions of the state space is divided into some smalldimensional state space. What’s more, heuristic techniques are used tocompletion function. It’s an effective way to reduce the space of statedimensions.Finally, in the Robocup simulation platform, this article designs the player’s decision making and function module, adds in dynamic potential field heuristicmodel to allows the player agent obtained the informatin of the location of thefootball field and the players to make adaptive judgement. Through forward treesearch algorithm, the player can make the plan to achieve collaboration withtheir teammates, this process uses the result of reinforcement learning of players’behavior, make the player to select the optimal strategy and to complete theprocess of decision-making. In this way, the ability of collaboration among theplayers has been improved.All of the proposed mechanisms and methods are applied to the RoboCup2D simulation platform team. In order to analysis the proposed algorithm, thisarticle uses the task of intercept and matches between different version of ourteam with related algorithm. The most important work is compare with otherteams in the form of the game. The results of the experiment were analyzed toverify the multi-agent decision-making methods which proposed in this paper isfeasible.