Research On Decision-Making Strategy Of Soccer Robot Based On Multi-Agent Reinforcement Learning

Nearly a decade,the research of distributed artificial intelligence has been paid more and more attention.And multi-agent system has also become the focus of research.The multi-agent reinforcement learning can be derived from the combination of multi-agent system and the reinforcement learning.In this paper,the soccer robot,which is a typical multi-agent system,is selected as the object of study.Problems of multi-agent reinforcement learning especially the multi-agent Q learning and the use in decision-making strategy of soccer robot are studied.Firstly,the research background and the current situation of this subject are introduced.The important theoretical and practical significance of our research are introduced.The detailed description of the soccer robot system and multi-agent system are also given.Secondly,the reinforcement learning of agent is studied and the basic concepts and principles are analyzed.The framework model of the reinforcement learning of agent,which is also called Markov decision process,is researched.The key factors,such as reward?strategy?value function and action exploration,are analyzed.Three kinds of reinforcement learning algorithms are introduced,which are TD algorithm,Q learning algorithm and Sarsa learning algorithm.Besides,the way of description of multi-agent system is presented and the characteristics and methods of reinforcement learning is discussed.Thirdly,the simulated annealing is applied to the action exploration of the reinforcement learning.So it provides a scientific solution for the exploration and utilization of action in learning.And then,the Q learning algorithm based on simulated annealing is presented.The learning space and the reward and punishment function of the robot searching target strategy of the soccer robot system which applied this algorithm are designed and analyzed and next the simulation results are verified by MATLAB platform.Finally,the role transformation and experience sharing of multi-agent reinforcement learning are analyzed based on the characteristics of soccer robot system decision making,and then the multi-agent Q learning algorithms based on role transformation and experience sharing is proposed.In the last,this method is applied to the local attacking strategy of soccer robot,and the action selection strategy of the main robot in the team is studied by using this algorithm,and the MATLAB platform is used to verify the strategy.