Researches On Optimal Control Of Nonlinear Systems Based On Approximate Dynamic Programming

The optimal control problem of nonlinear systems is the principal and diffi-cult domain. Since approximate dynamic programming was proposed, it has been regarded as an effective way to deal with the optimal control problem of nonlin-ear systems. Approximate dynamic programming, combining with neural networks, adaptive critic technique, reinforcement learning and dynamic programming the-ory, obtains the optimal control without the "curse of dimensionality" and then receives lots of attentions. So, it is of great importance on nonlinear optimal con-trol for the further research on the theory and algorithm of approximate dynamic programming. In this dissertation, based on approximate dynamic programming, many optimal control problems are investigated on nonlinear systems such as multi-objective optimal control, optimal tracking control, control of zero-sum differential problem, optimal control with delays, etc. The main research of the dissertation can be briefly described as follows:1. A new model-free optimal control method for a class of discrete-time non-linear systems with general multiobjective performance indices is proposed with un-known system dynamics. The proposed model-free Q-learning method aims to find out the increments of both the controls and states instead of computing the controls and states directly. Using the technique of dimension augment, the vector-valued performance indices are transformed into additive quadratic form which satisfies the corresponding discrete-time algebraic Riccati equation (DTARE). Both the action and critic networks can be adaptively tuned by adaptive critic methods without the information of the system model. The convergence property is guaranteed by a rigorous mathematical proof.2. For the first time, the infinite-time optimal tracking control problem for a class of discrete-time nonlinear systems is solved using greedy Heuristic Dynamic Programming (HDP) iteration algorithm. A new type of performance index is de-fined since the existed performance indices are very difficult in solving this kind of tracking problem if not impossible. Via system transformation, the optimal tracking problem is transformed into optimal regulation problem, and then the greedy HDP iteration algorithm is introduced to deal with the regulation problem with rigorous convergence analysis.3. An iterative adaptive dynamic programming (ADP) algorithm is proposed to solve a class of continuous-time nonlinear two-person zero-sum differential games. Using ADP technique to obtain the optimal control pair iteratively which makes the performance index function reach the saddle point of the zero-sum differential games. Stability analysis of the nonlinear systems is presented and the convergence property of the performance index function is also proved.4. Considering two-person zero-sum differential games when the saddle point does not exist, via iterative ADP algorithm, the mixed optimal control pair is ob-tained to make the performance index function reach the mixed optimum with rig-orous stability analysis of the nonlinear systems and the convergence proof of the performance index function.5. For the first time, an optimal control scheme for a class of affine nonlinear systems with time delays in state and control variables with respect to a quadratic performance index function is proposed using a new iterative adaptive dynamic pro-gramming (ADP) algorithm. By introducing a delay matrix function, the explicit expression of the optimal control is obtained using the dynamic programming theory and the optimal control can iteratively be obtained using the adaptive critic tech-nique. Convergence analysis is presented to prove the performance index function to reach the optimum by the proposed method.6. For a class of discrete-time nonlinear systems with state delay, using pseudo-linear technique which is the linear time-varying approximate technique, the system is transformed into a series of linear systems with state delay. Based on the classical theory of dynamic programming, the optimal control of the delay systems are ob-tained which satisfy the corresponding delayed algebra Riccati equation. Rigorous mathematical analysis is proposed to guarantee the stability of the system and the convergence property of the state under the delayed optimal control.