Design, Decision-making System Based On Reinforcement Learning For Soccer Robots

Robot Soccer is a focus of robot research in recent years. It involvesfields like robotics, artificial intelligence, intelligent control, computervision and so on. The strategy subsystem is the core of the whole RobotSoccer system, just like the brain of the robots. It is responsible for thecooperation of the robots. So the research of the strategy subsystem has asignificant meaning to the multi-robot and multi-agent fields.This thesis is based on the Mirosot 5v5 simulation contest. In orderto meet the Robot Soccer system's need on reactivity, adaptability,intelligence and learning ability, a dual strategy model based on improvedQ-learning is proposed. The whole strategy model includes the upperlayer: cooperation layer and the lower layer: movement control layer.By analyzing the character of the Robot Soccer match, a fuzzyclustering method is used by the upper cooperation layer to transfer thelarge-quantity system states to a few fuzzy states which reduces thenumber of the state greatly and speeds up the convergence of thealgorithm. At the same time, to avoid to converge to local optimal, anadaptive Q-learning algorithm is proposed by regulating threeQ-learning's parameters(learning rateα, discountγand temperatureT). So the global optimal action could be reached.To improve the reactive ability of the system, a reactive based agentstructure, which is quite different with the traditional method, is used todesign the lower movement control layer. It contains three types of agents:the defense agent, the attack agent and the sidekick agent.The whole strategy subsystem is designed to a Dynamic LinkedLibrary (DLL) program with C++language under Windows OS. Theeffectiveness of the improved Q-learning strategy and dual modelproposed in this dissertation is proved by competing with other teams onthe Mirosot 5v5 simulation platform.