Online Adaptive Optimal Controller Design For Continuous-Time Markovian Jumping Linear Systems

The online adaptive optimal controller design algorithms for a class of continuous-time Markovian jumping linear systems (MJLSs) are studied in this dissertation. A new online decouple technique, named Subsystems Transformation, is presented to separate the coupling relation from the subsystems of the MJLSs and into an explicit iteration equation form. Based upon Integral Reinforcement Learning (IRL) approach (i.e. Approximate Dynamic Programming (ADP) method), two novel policy iteration algorithms are developed to solve the H2/H∞optimal control problems of the continuous-time MJLSs. The structure of the new policy iteration is based on two-step iterations, i.e., Policy Evaluation and Policy Improvement.The greatest advantage of the proposed algorithms shows that it can solve the relevant optimal problems without using the exact knowledge of the systems dynamics. Moreover, from the viewpoint of computation, the convergence speeds of the developed new online policy iteration algorithms are much faster than some existing off-line computation iterative algorithms. The main works and contributions are shown in following five parts:1. Different with regular continuous-time linear/nonlinear systems, MJLSs have two components which are the jumping modes and continuous states, modeled by finite state Markov chains and differential equations, respectively. The variables not only associate with continuous-time domain but also depend on the probability space; thus the existing ADP approach cannot be applied to the optimal control problems of continuous-time MJLSs by a simple extension. In order to develop an algorithm to find the optimal controller for continuous-time MJLSs respect to model partial-free case, a novel decouple technique, which so-called Subsystems Transformation, is proposed. In sprit by the parallel iteration algorithm and by means of Subsystems Transformation technique, the coupling relation during the subsystems of MJLSs is separated into an explicit equation, i.e., Coupling Relation Update. The explicit equation plays a key role in developing online adaptive optimal controllers design algorithm. Some related contents have been presented in Chapter2and Chapter4.2. Chapter2investigates the online adaptive optimal control problem for continuous-time MJLSs based on a novel policy iteration algorithm. By utilizing Subsystems Transformation technique, the concerned MJLSs are reconstructed into a set of new coupled systems composed of N subsystems. The online policy iteration is used to solve the coupled algebraic matrix Riccati equations with partial knowledge regarding to the system dynamics, and the relevant optimal controllers are equivalent to the concerned MJLSs. Moreover, the convergence of the novel policy iteration algorithm is also proved. Finally, a simulation example is given to illustrate the effectiveness and applicability of the proposed approach in this chapter.3. In Chapter3, two computation iterative algorithms are studied to solve the coupled game algebraic Riccati equation (CGARE) associated with the optimal H∞control problems for continuous-time MJLSs. The two iterative algorithms are based on the framework of Kleinman iteration algorithm. At first, the direct parallel Kleinman iteration algorithm is proposed and the convergence of the iterative algorithm is established. Then, a more general iteration algorithm (named generalized parallel Kleinman iteration algorithm) is introduced by four different cases. Finally, a numerical example has been provided to demonstrate the effectiveness of the proposed algorithm.4. A new online policy iterative algorithm for solving H∞control problem of continuous-time MJLSs is shown in Chapter4. Based upon the direct parallel Kleinman iteration algorithm proposed in Chapter3, and by using Subsystems Transformation method, a set of linear subsystems which with the external disturbance input signal are obtained. Thereafter, the two-player zero-sum game and continuous-time CGARE are solved by means of a novel online ADP approach. The convergence of the novel policy iteration algorithm is also established. At last, simulation results have verified the applicability of the new online algorithm.5. Conclusions are given in Chapter5. And some suggestions on the research works related to the IRL-based optimal controller design problems for continuous/discrete-time Markov jump system also are given in this chapter.