Research On Classes Of Cooperative Optimal Control Algorithms Of Multi-agent Systems Via Reinforcement Learning

The cooperative optimal control problems of multi-agent systems refer to that when all agents complete a global control task by distributed cooperation,each agent also needs to optimize a predefined goal modeled as a performance index.In the research of cooperative control of multi-agent system,there exists the problem of imprecision or simplification of system models.It is investigated that reinforcement learning methods can be used to design optimal control algorithms which can be used to solve some op-timal control problems of different classes of systems with requiring part of the knowl-edge of system models or without requiring the knowledge of system models.There-fore,reinforcement learning based cooperative optimal control of multi-agent system has attracted scholars' research interest.The consensus problem of multi-agent systems,mainly refers to the consensus of the final state or output between agents by cooperation,can well describe the coopera-tive behavior of multi-agent systems.This paper mainly studies reinforcement learning based cooperative optimal control algorithms of multi-agent systems,so that the states or outputs of all agents achieve a consensus.Under this framework,based on the ex-isting research works of reinforcement learning in related problems,this paper makes further research on consensus problem and output regulation problem,mainly discusses how to design reinforcement learning based cooperative optimal control algorithms to find a distributed control policy for each agent with requiring part of the knowledge of system models or without requiring the knowledge of system models.The research contents can be divided into the following three parts:1.Design reinforcement learning based model-free algorithm to solve the fully co-operative optimal consensus problem of a class of linear continuous-time multi-agent systems with a leader system.Fully cooperative means that all followers need to optimize a same centralized performance function.Based on reinforce-ment learning methods and,an off-policy model-free Bellman equation is derived.Combining neural network technology,a reinforcement learning based off-policy model-free algorithm is proposed.It is proved by rigorous mathematical analysis that this algorithm can be used to solve the fully cooperative optimal consensus problem in a suboptimal way.Finally,a simulation result verifies the effective-ness of this algorithm.2.Design reinforcement learning based model-free algorithm to solve the fully co-operative optimal consensus problem of a class of nonlinear continuous time lead-erless multi-agent systems.Based on reinforcement learning methods and neural networks,a suboptimal reinforcement learning based off-policy model-free algo-rithm is proposed.Finally,two simulation results are given to verify the effec-tiveness of this algorithm.3.Design reinforcement learning based algorithms to solve the optimal cooperative output regulation problem for a class of linear discrete-time multi-agent systems.Firstly,a distributed observer is designed for each follower to estimate the leader's information.Then,a suboptimal distributed feedback controller is designed for each follower by using the observer and its state information.Then,a reinforce-ment learning based algorithm is proposed to find the optimal gains of the con-troller with only requiring part of the knowledge of system models.Finally,a simulation result is given to verify the effectiveness of this algorithm.