Synchronization Control Based On Model Reference Adaptive Critic Learning For Multi-Agent Systems

Since the multi-agent systems(MASs)have been extensively applied to the various fields,the research on the distributed cooperative control,a vital branch of MASs,has received compelling concerns.The distributed synchronization control is a fundamental research of the distributed cooperative control.Via local information exchange among agents,its objective is to design a controller or control protocol so that behaviors of MASs reach agreement in a sense.At present,in the existing distributed synchronization control studies,much attention has been paid to the simple dynamic systems,such as single or double integrators.In these studies,the control protocol is designed with requirement of a prior knowledge of MASs.In the practical scenario,the real MASs are always complex nonlinear systems,including external disturbance and unknown dynamics.Moreover,it is also desired that their closed-loop control systems have the flexible cooperation capacity and optimal system performance.In these cases,the main obstacle in designing the control protocol lies in that the coupled Bellman equation is difficult to solve.And therefore,it is intractable to achieve the distributed synchronization control of the nonlinear MASs.This limits the application of the existing studies in the complicated scenario.To overcome these deficiencies,in view of the self-learning property of adaptive critic designs(ACDs),the ACDs-based distributed synchronization control for complex nonlinear MASs is studied in this dissertation,whose primary contents are summarized as follows.(1)Model reference adaptive critic learning control for unknown single-input nonlinear systemsThe neural network approximation errors arising from ACDs in the model reference adaptive control are taken into account for single-input unknown nonlinear systems with the persistent bounded disturbance.To tackle this problem,A model reference adaptive critic learning control approach is proposed to allow behaviors of the nonlinear systems to flexibly follow those of the prescribed reference model online in real time.Such an approach ensures optimality of the closed-loop control systems and implementation of the chattering-free sliding control,and hence has the robustness to the uncertainties lumped with the approximation errors and disturbance.(2)Supervised model reference adaptive critic learning control for unknown multi-input nonlinear systemsOn the basis of the model reference adaptive critic learning control approach,a supervised model reference adaptive critic learning control approach is presented by introducing a supervisor into the model reference adaptive critic learning control.In such an approach,multi-input unknown nonlinear systems are considered and the assumption on the bounded drift dynamics of the systems is removed.This approach not only guides the leaning or training of the actor-critic network,but also produces a compensation control law.And therefore,the closed-loop control systems are robustness to the uncertainties lumped with the approximation errors and disturbance.(3)Optimal synchronization control for partially unknown nonlinear MASsSince the states of neighbor agents are introduced in the local neighbor tracking error dynamics for nonlinear MASs,solving the coupled Bellman equation becomes more difficult,especially for partially unknown nonlinear MASs.This is also a key issue in designing the optimal control protocol.This dissertation establishes a hierarchical and distributed optimal synchronization control framework.In the decentralized model reference adaptive control layer,a neural-network adaptive control method is utilized to allow the behaviors of all the agents to track those of their corresponding reference models.In the distributed optimal synchronization layer,for the reference MASs consisting of the reference models and a leader,a distributed model reference adaptive critic learning synchronization control approach is proposed via a distributed value iteration learning method.This approach realizes the reference MASs being in Nash equilibrium.That is to say,the behaviors of the reference MASs reach agreement with optimum,which means that the optimal synchronization control for MASs is achieved.(4)Optimal synchronization control for unknown nonlinear MASsFor completely unknown nonlinear MASs,this dissertation establishes a new hierarchical and distributed optimal synchronization control framework,based on the model-free distributed model reference adaptive critic learning approach.In the decentralized model reference adaptive control layer,a similar-offline neural-network adaptive control method is designed to relieve the computation burden in the optimal synchronization control.In the distributed optimal synchronization layer,this dissertation proposed a distributed reference police iteration learning method to obtain the Nash equilibrium solution of the coupled composite nonlinear Bellman's equations.A model-free distributed model reference adaptive critic learning synchronization control approach is further developed to guarantee that the MASs reach the optimal consensus with no prior knowledge of the system model.(5)Optimal synchronization control of unknown nonlinear MASs with an active leaderBased on a distributed supervised model reference adaptive critic learning approach,another hierarchical and distributed optimal synchronization control framework is established for unknown nonlinear MASs with an active leader and persistent disturbance.In the decentralized model reference adaptive control layer,the supervised model reference adaptive critic learning control approach is applied to suppress the uncertainties lumped with the approximation errors and disturbance.In the distributed optimal synchronization layer,this dissertation develops a distributed supervised model reference adaptive critic learning synchronization control approach.This approach guarantees MASs being in Nash equilibrium,and therefore the optimal synchronization control problem for MASs is solved and the flexible cooperation capability is also guaranteed.