Data-driven Consensus Control For Multi-agent Systems With Time Delay

In recent years,multi-agent systems have gradually replaced single-agent systems to perform various complex tasks because of their outstanding abilities in function,system design and robustness.It has been widely used in the fields of smart power grids,intelligent logistics,smart factory,military field and so on.Cooperative control of multi-agent systems is one of the key research directions.Consensus problem,as one of basic problems of cooperative control for multi-agent systems,has also been paid close attention.In most practical multi-agent systems,the time delay is an inherent feature and often occurs during the process of information exchange between neighbor agents.Time delay can damage the systems performance and even prevent multi-agent systems from converging to consensus.At the same time,the practical multi-agent systems usually have the characteristics of unknown dynamics and uncertain model,so it is difficult to establish an accurate mathematical model.Traditional methods such as proportional intergration control,fuzzy control,model predictive control and so on can not be applied.Therefore,based on Q-learning and adaptive dynamic programming,this paper studies consensus problems of a series of multi-agent systems with time delay.The main research contents of the paper are as follows:Firstly,a Q-learning algorithm based on adaptive dynamic programming is proposed to solve the tracking control problem of discrete time single-agent systems with time delay.In the absence of the system dynamics,linear quadratic tracker is transformed into linear quadratic regulator.Then,according to the Bellman optimal principle,the modeless controller is designed based on reinforcement learning and least squares.Thus,the optimal tracking control is achieved for the single-agent systems with unknown dynamics.Secondly,for a class of discrete-time systems with multiple state delays,a Q-learning consensus control algorithm based on policy iteration is proposed to solve the optimal consensus control problem.It avoids solving the coupled Hamilton-Jacobi-Bellman equation directly.All the followers are synchronized with the leader in the multi-agent systems and the performance index is optimal.First,by means of coordinate transformation,the minmum state delays equivalent system of the tracking error system with multiple state delays is deduced and the equivalent conditions of two systems are derived.Then,a Q-learning algorithm based on data is proposed.And policy iteration and least square method is adopted to obtain the optimal control solution.Relevant proof and simulation are provided,which prove the convergence and effectiveness of the proposed Q-learning algorithm based on policy iteration.Finally,a data-driven consensus control algorithm based on adaptive dynamic programming is proposed for the linear discrete multi-agent systems with multiple delays and disturbances.Firstly,the minmum multiple delays equivalent system of the tracking error system with multiple delays is deduced.Then,a state estimator is constructed for the tracking error of the multi-agent systems.After that,the new data-based state equation and Bellman equation are exactly given.Then,a data-driven consensus control algorithm based on adaptive dynamic programming is designed.The algorithm can minimize the performance index and ensure the optimal consensus of the multi-agent systems,using only the measured system input and output data.Finally,simulation results verify the validity of the proposed algorithm for multi-agent systems with multiple delay and disturbances.