Adaptive Optimal Control For Continuous Nonlinear Systems Based On Adaptive Dynamic Programming

In recent years, an adaptive dynamic programming (ADP) method that combines the dynamic programming, reinforcement learning and adaptive techniques has been suggested to solve the optimal control problem of nonlinear systems. Recently, this method has become a hot research topic in the community of intelligent control and computational intelligence.Based on ADP structure, this paper studies the optimal control problem of nonlinear systems with completely unknown dynamics. For the unknown dynamic systems, a new identifier-actor structure is suggested, where the classical actor NN is not needed. Moreover, novel adaptive laws based on the parameter estimation error are used to online estimate the identifier and actor NN weights. Furthermore, the proposed adaptive method is applied to solve the optimal tracking control problem. The main work of this thesis can be summarized as follows:(1) Identification of unknown system dynamics based on neural network and adaptive laws with parameter estimation error. This work uses a new adaptive law based on the parameter error to estimate the NN weights online, which can guarantee that the estimated parameters converge to their true values with enhanced robustness.(2) Optimal regulation control for nonlinear systems based on the adaptive dynamic programming. In this new framework, the actor NN used in traditional identifier-critic-actor structure is not needed, which could greatly reduce the computation cost.(3)Optimal tracking control for nonlinear systems based on the adaptive dynamic programming. Based on the above identifier-critic structure, this section further studies the optimal tracking control of nonlinear systems. The overall optimal tracking control consists of a steady-state control and an optimal control to regulate the tracking error. All these two control actions can be online obtained by using the identified dynamics and critic NN. Finally, we also suggest a new augmented system including the reference model and tracking error system to design the optimal tracking control, where the feedback and feedforward parts can be obtained simultaneously.(4) Finite-time adaptive laws for optimal control design. The idea of sliding mode control is further adopted to develop modified adaptive laws, which can guarantee finite time convergence of the estimated weights of the identifier and the critic NNs. In this case, faster convergence rate is obtained.Finally, extensive simulation and experiments based on nonlinear systems and a 3 degree of freedom (DOF) helicopter system are carried out to verify the validity of the proposed methods. Thus, the obtained results may provide potential technical solutions for the optimal control designs for more complex systems.