Distributed Optimal Consensus Control For Multi-agent Systems Based On Iterative Reinforcement Learning

Distributed consensus control for multi-agent systems(MASs)is a research direction that has received a lot of attention,because it is applied in many fields such as space surveillance,traffic signal control and unmanned aircraft formations.This research direction focuses on how to design a control strategy relying only on information about themselves and their neighbours for each agent so as to work collaboratively.Depending on the presence or absence of leaders in the system,this research direction can be divided into leaderless consensus controls and consensus controls with leaders.For consensus control with leaders,depending on the number of leaders,it can be further divided into track consensus controls with only one leader and containment controls with multiple leaders.There have been many stuies on distributed consensus control,however,in the process of information exchange of a realistic multi-agent system,due to the limited bandwidth in shared communication networks,i.e.the limited amount of data transmission per unit time,when the communication transmission frequency is too high,it can cause data transmission delays,which leads to communication delays between agents;at the same time,due to the limited computational capacity of the agents themselves,when the controller update frequency is too high,it can cause calculation overload,resulting in input delays within the agents.Therefore,time delay is often a inevitable factor in solving the problem of distributed consensus control for multi-agent systems.In this thesis,the problems of distributed optimal consensus control for multi-agent systems based on iterative reinforcement learning are studied,combining algebraic graph theory,adaptive distributed observer design,networked predictive controller design,neural network approximation,optimal control and reinforcement learning to estimating the state and compensating for the delays,and the thesis verifies the feasibility of the control strategy by using matlab for simulation.The main work in this thesis consists of three aspects as follows:1.For the distributed optimal state consensus control problem of a homogeneous multi-agent system with communication delays,a distributed optimal control strategy based on networked predictive controller is proposed.First,the networked predictive control based compensation mechanism is presented,where distributed observer is used to estimate the state of each follower.Then,a distributed optimal state consensus control strategy is designed,and Lyapunov function is constructed to prove that followers can reach state consistency with leader.Then,a value iteration algorithm is established to learn the solutions to the HJB equations online,then it is used in a critic-action neural network structure.Finally,simulation is made by matlab to show the feasibility of the theoretical methods.2.For the distributed optimal output consensus control problem of a heterogeneous multi-agent system with input delays,a distributed optimal control strategy based on adaptive distributed observer is proposed.First,After adopting the idea of discreting and transforming the model,a continuous-time system with input delay is converted into a discrete-time system without input delay.Then,when the model information of the leader is not gotten,a adaptive distributed observer is designed to estimate the state of leader for each follower.Then,a distributed optimal output consensus control strategy is designed,and Lyapunov function is constructed to prove that followers can reach output consistency with leader.Then,an Q-learning algorithm is established to solve the ARE online without requiring dynamics of all agents.Finally,simulation is made by matlab to show the feasibility of the theoretical methods.3.For the distributed optimal state containment control problem of a heterogeneous multi-agent system,a distributed optimal state containment control strategy based on adaptive distributed observer is proposed.First,a adaptive distributed observer is proposed to estimate the states of leaders and construct the state trajectories insider the convex hull of the leaders.Then,a distributed optimal state containment control strategy is designed,and Lyapunov function is constructed to prove that the states of followers can converge to the convex hull spanned by the states of the leaders.Then,when the model knowledge of the leader and the agents is not gotten,an model-free reinforcement learning algorithm is established to solve the ARE online.Finally,simulation is made by matlab to show the feasibility of the theoretical methods.