Model-free Adaptive Dynamic Programming And Its Applications On Multi-agent Systems

Intelligent control systems,which imitates human's process of learning to acquire knowledge,can efficiently make up deficiencies in classical control theory and develop novel ideas to overcome technical problems.Adaptive dynamic programming(ADP),a method that combines merits of neural networks,reinforcement learning and adaptive critic designs,is able to avoid the 'curse of dimensionality' and obtain the approximate optimal control policy.This thesis investigates some key issues in ADP theory,including the convergence of off-line and on-line learning algorithms,and the closed-loop stability.Moreover,in this thesis,ADP is applied to the cooperative control of multi-agent systems(MASs).The main contributions of this dissertation can be briefly listed as follows.(1)For the optimal regulation problem of continuous-time nonlinear dynamical systems,three fundamental problems in optimal control theory are refined based on iterative optimization algorithms.On this basis,a novel Hamiltonian-driven framework is presented to reveal the necessary and sufficient condition that guarantees the convergence of iterative ADP algorithms.(2)For the robust stabilization problem of discrete-time dynamical systems,it is shown that this problem can be translated into an equivalent optimal control problem.The sufficient condition that guarantees the problem translation equivalence is given theoretically.Furthermore,based on the on-line measured data,a data-driven and model-free ADP for discrete-time dynamical systems is presented to avoid the requirement of complete exact system dynamics.Convergence proof and stability analysis are also provided.(3)For the output synchronization problem of heterogeneous linear MASs with a leader that contains unknown control input,a model-free ADP algorithm is presented to obtain the distributed optimal output synchronization control laws.Note that in most existing results on output synchronization problem of MASs,only asymptotic stability of synchronization error is considered.In this dissertation,optimality is explicitly imposed to the distributed output synchronization control by considering the performance of transient output synchronization error.(4)For the containment control problem of heterogeneous linear MASs with multiple leaders,a model-free ADP algorithm is presented to obtain the distributed optimal containment control protocols.Note that in most existing literatures,the distributed cooperative control protocols depend on the eigenvalue of the Laplacian matrix,which is a global information of the overall systems.However,in this thesis,this requirement is avoided by a distributed compensator design which does need any information about the Laplacian matrix.Therefore,this method is fully distributed in essence.Convergence proof and stability analysis for the presented distributed optimal containment control are given in the end.