Researches On Adaptive Dynamic Programming Theory And Its Applications On Some Classes Of Uncertain Nonlinear Systems

With the development of science and technology,the scale and complexity of engineering control system is increasing.The uncertainties may degrade the perfor-mance of a controlled system or even make the closed-loop system unstable when they are not appropriately accounted during the controller design stage,resulting in unexpected losses.Therefore,it is very important to improve the safety and re-liability of the controlled system.Due to the complexity of nonlinear systems,the development of control theory is not perfect,and the corresponding control methods of uncertain nonlinear systems are very limited.And most of the available results are mainly about system stability.The system performance optimization problem is rarely considered.Integrating with neural network,reinforcement learning and adaptive evaluation design,the adaptive dynamic programming(ADP)method is presented to solve the optimal control problem of complex systems,avoiding the "dimension disaster"problem in the dynamic programming algorithm.In this paper,based on ADP theory,combined with the theory of optimal control,game theory and sliding mode control,optimal tracking control problems of nonlinear systems and the control problems of uncertain nonlinear systems are studied.The main contents are listed as follows:(1)For multi-player nonlinear systems with input disturbances,the robust control problem is transformed into the multi-person non-zero sum problem.By ana-lyzing the robust characteristic of Nash solution of nominal nonlinear nonzero-sum game with predefined cost functions,a pair of robust control policies is constructed.Sufficient conditions for the existence of robust control strategy are derived through lyapunov method.The single-network ADP algorithm is employed to solve the coupled Hamilton-Jacobi equations,where only requires to online tune the weights of critic neural networks(NN)for each player.By utilizing Lyapunov theory,the NN weight estimation errors are proved to be uniformly ultimately bounded.(2)The decentralized tracking problem for nonlinear large-scale interconnected systems is firstly transformed to optimal regulation problem for augmented subsystems composed of the error system dynamics and the command gen-erator dynamic associated with each isolated subsystems,in order to design the novel decentralized adaptive tracking control strategy.A single critic NN-based adaptive dynamic programming algorithm is used to solve the nonlinear HJB equation,which is implemented online in real-time.By employing a aux-iliary term in the critic NN weight updating law,there is no requirement for initial admissible control in the proposed algorithm.Based on Lyapunov the-ory,stability analysis of the closed-loop augmented subsystem is performed to show that all tracking errors and NN weight approximation errors are uni-formly ultimately bounded.(3)A novel constrained composite sliding mode controller is studied based on IS-M and ADP theory for uncertain nonlinear systems with input saturation.The proposed controller consists of two parts,the discontinuous control part ensures that the system maintains on the sliding mode surface,while the con-tinuous control part ensures that the closed system states move along the opti-mal trajectory.A non-quadratic functional is used to deal with the saturation input problem,and the constrained optimal control for nominal nonlinear sys-tems is approximated by using an online approximate learning algorithm with actor-critic NN framework.Lyapunov techniques are used to demonstrate the uniform ultimate bounded convergence condition for closed-loop nominal system and the weight estimation errors.(4)By combining ISM with nonlinear H? optimal control theory,a novel nonlin-ear control scheme is proposed for uncertain nonlinear systems with actuator faults and unmatched disturbances.The compound control strategy consists of two parts:one part is a discontinuous sliding mode controller properly de-signed for reducing the effect of the actuator faults and the separated matched disturbance component,and the other part is a nonlinear H? control to be designed for attenuating unmatched disturbance component.The nonlinear H? control is obtained by using an online policy update algorithm based on actor-critic-disturbance NNs to approximately solve HJI equations for equiva-lent sliding mode dynamics.Lyapunov techniques are used to demonstrate the convergence of the NN weight errors in the sense of uniform ultimate bounded.(5)Based on ISM and ADP theory,a novel optimal guaranteed cost sliding mode control is designed for uncertain constrained-input nonlinear systems.The compound control strategy consists of two parts:one part is a discontinuous controller designed for deriving the system moves on the sliding surface;the other part is continuous controller designed for guaranteeing the system sta-bility and minimum performance up bound of the closed-loop system.The ADP algorithm based on single critic NN is applied to solve the HJB equation without requiring the initial admissible control.Lyapunov techniques are used to demonstrate the convergence of the uncertain closed-loop system and NN weight errors in the sense of uniform ultimate bounded.