Research On Online Learning Algorithm Of Multi-agent Graphic Game

Multi-agent graphical game is an important research topic of modern control theory,which has been studied by many scholars at home and abroad.For each agent,a reasonable performance index that only depends on the local information available to each agent is defined.Although the optimal control of each agent can be obtained by solving the Hamilton-Jacobi-Bellman(HJB)equation,the equations are coupled in the multi-agent graphical game,so solving the HJB equation in the tradi-tional way is often very difficult or even impossible to solve.Reinforcement learning is a subfield of machine learning.The combination of reinforcement learning and control theory can often solve some problems that are difficult to solve with tradi-tional control theory methods.Some scholars have proposed some online iterative algorithms based on reinforcement learning by combining reinforcement learning and control theory.These algorithms often use onlire iterative methods to solve the HJB equation without the need for system matrices,avoiding the process of the tradi-tional method solving complex HJB equations.In recent years,many research results based on the control problem of reinforcement learning methods have been put for-ward,such as the zero-sum game problem raised,discrete time multi-agent graphical game problems,continuous time multi-agent graphical game problems,and multi-agent graphical game problems with input restrictions,etc.Based on the existing research results,this paper mainly studies the discrete time multi-agent graphical game with input constraints.In this problem,in order to obtain the optimal control of each agent,it is necessary to solve a series of coupled HJB equations,but it is difficult to solve the HJB equation using traditional methods,and the related game problem will becomes more complex if the policy is constrained.In this paper,a performance index that only depends on the local information available to the agent is defined.Based on this performance index,an online iterative algorithm that is used to find the online iterative solution of a dynamic graphic game with input con-straints is proposed.In fact,this algorithm finds the solution to the Bellman equation online.This solution applies a distributed strategy iterative processing.Each agent only uses the local information that it can obtain.During the implementation of the algorithm,each agent uses two neural networks to fit the value function and control strategy respectively.It can be proved that under certain conditions,each agent up-dates its strategy through the algorithm,and eventually all agents will form a Nash equilibrium.