Research On Reinforcement Learning Algorithm And Equilibrium Of Multi-Agent Game

In practical applications,artificial intelligence and multi-agent systems will encounter scenarios such as unmanned driving,drone formation,robot confrontation etc.,where agents interact with the environment and agents interact with agents.These scenarios can be studied under the framework of Markov games or Random games.Focusing on the problems in the multi-agent game model based on reinforcement learning method and environment interaction to find Nash equilibrium multi-agent reinforcement learning algorithm,based on equilibrium theory to calculate Nash equilibrium group intelligence algorithm,the following research is carried out:In order to improve the problems of multi-agent Nash Q learning algorithm,such as poor adaptability,harsh conditions,complex calculations and no general method to update strategy value,the idea of algorithm improvement is proposed.First,the joint action vector simplification algorithm and the parameters are introduced,the state-behavior value function is approximated by the parameter,the training target is transformed,and the value function update equation of the parameter approximation is given.Secondly,the convergence and feasibility of the algorithm are theoretically analyzed.Finally,The effectiveness of the algorithm is verified through experiments.The simulation results show that the multi-agent reinforcement learning algorithm based on parameter approximation can make the agent achieving 100%Nash equilibrium,and can improve the performance of the algorithm,simplify the algorithm complexity,and can converge faster than the traditional Nash Q learning algorithm.Aiming at the difficulty of Nash equilibrium calculation in matrix games with a large number of agents or slightly higher dimensions,the existing Nash equilibrium solution theory is analyzed to convert the Nash equilibrium problem into a single-objective optimization problem that can be solved by swarm intelligence algorithm.There are some problems in calculating Nash equilibrium.For example,the accuracy is not high and the iteration is cumbersome.In order to improve the existing particle swarm algorithm for calculating Nash equilibrium,an improved particle swarm algorithm is proposed.By analyzing the parameters and designing the correction scheme of the parameters,the improved algorithm steps are given;By using some test functions to verify the feasibility and effectiveness of the improved algorithm,the general matrix game is transformed to solve the Nash equilibrium,and the improved algorithm is given to solve the Nash equilibrium process and experimental verification.These results show that the improved algorithm is feasible to calculate the Nash equilibrium.The algorithm can not only effectively solve the Nash equilibrium value,but also improve the calculation accuracy and algorithm performance.