Adaptive Optimal Tracking Control And Application To Robotic Systems

Optimal control has always been a key focus in the control community during the past decades.Optimal control aims to seek a class of conditions and/or control methods that can be used to minimize a predefined performance index function of the control system,and it has been widely used in realistic applications.Generally,the optimal control problems are always offline to solve,where one can derive the optimal control policy by solving the algebraic Riccati equation(ARE)for linear systems or calculating the Hamilton-Jacobi-Bellman(HJB)equation for nonlinear systems offline.Dynamic programming(DP)has been used an effective tool for determining the solution of optimal control problems,which has been developed based on the Bellman's principle of optimality.However,the DP technique is generally difficult to solve optimal control for nonlinear systems online due to its well-known “curse of dimension” issue.To tackle the difficulties encountered in applying DP,adaptive dynamic programming(ADP)algorithm was recently proposed,which combines the advantages of DP,reinforcement learning(RL)and adaptive control techniques.By constructing a system called “critic” to estimate the optimal cost function,the approximate optimal control solution can be derived based on the approximated cost function.On the other hand,the conventional methods to solve optimal control problems requires partially or completely knowledge of the system dynamics.This fact makes the derived HJB equations for time-variant nonlinear continuous-time systems with completely unknown dynamics intractable,and thus restricts the application of optimal control in practice.In recent years,with the development of artificial intelligence technology,it is becoming emerging and possible to solve optimal control problems online by applying adaptive algorithms to address uncertainties of nonlinear systems.In this thesis,a novel adaptive algorithm driven by parameter estimation errors is employed and depending on the “identifier-critic” NN structure a novel adaptive dynamic programming algorithm will be constructed to solve optimal tracking control problems.By using adaptive algorithm the unknown parameters can be estimated online and converge to their real value rapidly.For the purpose to extend the application bound of the optimal tracking control algorithm,we consider three distinct systems in this thesis:(1)To solve the linear quadratic tracking problem with completely-unknown system dynamics,an adaptive identifier based vectorization operator and Kronecker product operator is employed to obtain the unknown dynamics.Then by using system augmented technology,the feedback and feedforward of the optimal control policy can be obtained simultaneously.The optimal tracking control policy can be derived online under the iteration algorithm.(2)For nonlinear optimal tracking control problems with completely unknown dynamics,the unknown system dynamics can be estimated by an adaptive identifier based NN.Then the augmented system is constructed that comprises the tracing error dynamics and the command trajectory and the modified augmented tracking HJB equation can be formulated.By defining an augmented system state consisting of the system dynamics and the reference trajectory,a compact optimal control action is derived online depending on the adaptive dynamic programming structure.(3)To extend the application scope of the proposed algorithm,we investigates the optimal tracking control problem for robotic systems with partially unknown dynamic.By employing an unknown input observer(UDE),we can derive the unknown dynamics.Then the optimal tracking control policy can be obtained online based on ADP framework.Note that,a discounted factor is considered for all of the cost function(performance index function)in this thesis.Moreover,simulations based on robotic systems are provided to show the feasibility of the proposed approach.Finally,experiments were also carried out by applying the proposed algorithms to a SCARA experimental platform built in the laboratory.The experimental results validate the proposed algorithms and thus show its potential application in practical engineering systems.