A Hierarchical Reinforcement Learning Method With Variable Learning Rate For Soccer Robot Multi-agent Adversarial System

Based on soccer robot simulation as its research platform, this paper studies the learning of high level strategy of multi-agent adversarial system. According to the study and analysis of soccer robot system, fuzzy method is introduced to describe the complex state space of soccer robot. This paper applies MAXQ, a multi-agent hierarchical reinforcement learning method, to the learning of attacking strategy of soccer robot. Moreover, a learning principle with variable learning rate is integrated into the MAXQ algorithm. The ultimate algorithm is validated to be suitable for multi-agent adversarial system.Multi-agent system is one of the research hotspots in the area of Artificial Intelligence. Compared with single agent system, multi-agent system has the properties of collective intelligence and sociality which are more conformable to the characteristics of many applications. Soccer robot system is one of the standard problems of this area. All the hotspots in multi-agent, such as agent, collaboration, communication, etc, are included in soccer robot simulation platform. This paper focuses on the hierarchical learning of high level strategy of soccer robot system. The attacking problem is abstracted for this paper.The description of environment is a precondition of agent learning. Agents in multi-agent system are always in complex, dynamic and not self-contained. The description depends on the specific case. This paper utilizes fuzzy method to describe the state space of the soccer player. With a reasonable state number, the state is effectively described.MAXQ, a hierarchical reinforcement learning method for multi-agent system, is proposed in recent years. It improves the reinforcement learning method to adapt for the multi-agent learning environment. This paper adopts MAXQ algorithm to the attacking strategy of soccer robot system. Experiment shows that this algorithm can learn attacking skills.Soccer robot system is an adversarial system. WOLF (Win or Learn Fast) is a newly proposed learning principle which is proven to be suitable for adversarial system. Using this principle together with MAXQ, a new method is proposed. Experiment result shows that this method has some good characteristics towards adversarial system.