Study On The Improved Average Reward Reinforcement Learning Algorithm Based On Performance Potentials

Reinforcement learning is an important learning control method for solving the problem of learning control in the field of artificial intelligence. The robot soccer simulation game, with the characteristics of dynamic real-time, distributed control, cooperation and confrontation in uncertain environment, is a benchmark problem for Multi-agent System research. Also it is a significant research for the progress of artificial intelligence, control decision and intelligent robotics. However, towards the disadvantages of conventional reinforcement learning algorithms in solving agents’ strategies, such as slow convergence, leading to environmental uncertainty and parameter sensitivity, this thesis proposes the corresponding improvements. The main contributions and achievements of this thesis are given below:Firstly, we give a fundamental introduction about reinforcement learning and performance potentials with their development progress, basic theory and typical algorithms. After that we will analyze their advantages and disadvantages respectively.Secondly, using conventional average reinforcement learning in solving agent’s personal skills may cause some problems with slow solution speed, local optimal, etc. For this reason, we employ G-learning algorithm for the off-line training of kick skill to improve the performance of agent’s skill. Then we perform a series of experiments and the results which show that G-learning is superior to conventional reinforcement learning both for convergence speed and successful rate of kick skill.Finally, we propose a multi-agent collaboration algorithm based on G-learning which not only need to solve the state space in multi-agent system, but consider about the learning and reward problems. In this part, we apply the improved algorithm to the Keepaway platform which enable the team to get a better performance.In conclusion, this work is achieved based on GDUT TiJi, a2D soccer simulation team of RoboCup. After the code programming, we also take part in RoboCup2013WorldCup and RoboCup China Open2013with the satisfactory results.