The Research Of RoboCup Simulation2D System

Being the largest intelligent football match in recent years, RoboCup (robot World Cup) has the largest number of people involved. And the Simulation2D match, whose problem of multi-agent system cooperation is one of the hot research topics, is ranked as one of the most traditional matches in the World Cup. It offers the researchers a standard platform as well as orders and operations. By doing so, RoboCup league encourages its researchers to obtain the solutions to problems concerning artificial intelligence and Multi-agent System through a variety of methods. It also enables them to test and practice via this platform.Based on the platform provided by RoboCup league, every school and research institute start researching study algorithm, which enables each agent to acquire the learning ability, thus guaranteeing the invincible status of the team created by schools and research institutes. And Q method is one of the widely used study algorithms.The improvements of the team based on the Q-Learning are as follows:1、Owing to the dynamic course and the multi-agent environment, MDP decision making process is not capable of reacting properly to the changes in the court. We adopt the Stochastic Game to replace the MDP.2、According to the Stochastic Game, complicated field environment will be discussed in three cases separately (agent controls the ball, our team control the ball, the other team control the ball), each case has a different state--action pair.3、By adopting supplementary control method and dividing the court precisely, Reward values of the key parameters in Q learning are calculated to ensure the continuity and monotonicity of reward value. At the same time, the Q value will be updated through the parameters and the improved algorithm, making the robot learning effect better.Experiments show that using this algorithm can strengthen the cooperation ability among the players obviously. Therefore the team’s overall attack and defense ability can be enhanced. However restricted by computer hardware, software and computing capacity, this algorithm can not be used in all agents; otherwise it is impossible to ensure a real-time game. Based on the theory, DreamWing2D of Anhui University team achieved success in the RoboCup China Open Simulation in2D, which proves the practicability of this algorithm.