Cooperation Mechanism Of Simulation 2D Soccer Robot Based On Reinforcement Learning

Based on the agent learning,this paper uses RoboCup2 D as the experimental platform to study the multi-agent reinforcement learning and collaboration problems.The research contents are as follows:In the problem of single-agent control strategy,a method of simulating soccer robot ball control based on Sarsa(?) algorithm is proposed.Firstly,Sarsa(?) learning algorithm is added to the ball-handling player to optimize the ball-control action.Then,through the analysis of the characteristics of the Keepaway keeper-taker control model,the state variables of the ball-handling player are reasonably divided according to whether the ball-handling player has the ball.The right defines the action function and the reward and punishment value.The experiment shows that the reinforcement learning of the Sarsa(?) algorithm makes the ball control time of the ball control player prolonged in the Keepaway,and the success rate of the ball control is improved.Aiming at the problem of offensive collaboration of multi-agents,a communication-based Sarsa(?) algorithm to broadcast the mechanism of agent messages in real time is designed.Firstly,the Sarsa(?) algorithm of communication is added to the player's offensive collaboration,and the current state-action pair message of the agent is broadcasted in real time,which improves the communication efficiency between the multi-agents.Secondly,the state variable of the offensive player is updated according to the distance and angle between the players,and calculate the action function and reward value of the offensive player with the goal of scoring goals and enhancing the efficiency of collaboration.Compare and analyze the reinforcement learning process with communication and no communication Sarsa(?) algorithm,and verify that the communication Sarsa(?) algorithm can strengthen the offensive cooperation of multi-agents and improve the efficiency of offensive collaboration.In the attack and defense cooperation problem of multi-agent,a collaboration balancing system based on Q-learning(?) algorithm is designed.The Q-learning(?) algorithm is applied to the multi-agent collaboration to speed up the learning convergence.Then,based on the stadium regionalization,the state variables of the agent are divided,and the action function is decomposed into offensive and defensive collaboration.The action function of the defensive collaboration is composed of the general player and the goalkeeper,according to both the agent's target of goal and the positional change of the ball as the consideration of the reward value.The experimental data proves that the Q-learning(?) algorithm converges quickly and can effectively improve the team's collaboration ability of attack and defense.