The Research And Implementation Of Agent Intelligent Decision Based On Q Learning

Research on the theory and application of multi-agent system(MAS) has become a hotspot of artificial intelligence. The Robot World Cup (RoboCup) is a typical MAS with characters such,as dynamic environment, the coexistence of cooperation and competition among several agents, limited communication bandwidth, and the noisy environment. Based on this general test platform, various theories and algorithms of MAS can be researched and evaluated.Reinforcement learning is an unsupervised learning technology, by which the agent can find optimal behavior sequence and perform on-line learning. So reinforcement learning is recognized as an ideal technology to construct intelligent agent.In this dissertation, the approach for agent to make intelligent decision is studied based on Q learning method, including the design of decision framework and the implementation of individual skill and team cooperation.Considering the complexity of agent decision task in RoboCup, layer learning based on decision framework is designed. The framework divides the full decision task into several layers from high-level to low-level. Machine learning (ML) techniques are used in each layer and the higher layer is implemented based on the lower layer. The layered decision framework combined with ML techniques can overcome the limitation of hand-coded implementation.In order to improve the intelligence of individual skills, Q learning method is adopted to train basic skills of ball kicking and dribbling. Since the state space in RoboCup is continuous, CMAC network is used to realize the state generalization in Q learning.For the learning problem of agent team cooperation, the basic Q learning algorithm is extended by introducing the concept of leading agent and setting the reward assignment policy among several learning agents. Two typical team cooperation problerms such as ball pass and 2 vsl cooperation are solved by the extended Q learning algorthm.All the experiments are made under RoboCup simulation platform. The results have proved that Q learning method can effectively improve the intelligence of agent decision in complex domain.