Study On Stochastic Control For Discrete-time Time-delay Systems

Optimal control and stabilization problems are fundamental in control theory and applications. For deterministic systems with delays or stochastic systems with-out delay, these problems have been extensively studied and fruitful results have been derived. However, for stochastic systems with delays, they are faced with huge challenges such as failure of separation principle, adaptiveness of control and infinite dimension and so on. These challenges make the optimal control and stabilization problems for stochastic systems with delays long-standing and difficult problems.This thesis is concerned with linear quadratic regulation (LQR) and stabiliza-tion problems for discrete-time stochastic systems with delays. Contributions and novelty of this thesis are as follows. Firstly, finite-horizon and infinite-horizon LQR problems and stabilization problem for discrete-time stochastic systems with a sin-gle input delay are considered. It is shown for the first time that the optimal con-troller is a predictor form in both the finite-horizon case and the infinte-horizon case. It also presents optimal feedback gains in terms of Riccati-ZXL equations. More importantly, a necessary and sufficient stabilization condition for the system under consideration is established for the first time. In addition, the technique to construct relations between the costate and the state supplies new ideas for the solving of gen-eral delayed forward backward stochastic difference equations. Also, our approach to constructing Lyapunov functions based on the optimal cost function is helpful to the study of other stabilization problems for time-delay systems; Secondly, a neces-sary and sufficient stabilization condition for a class of discrete-time systems with multiplicative noises and multiple input delays in the control variable is presented in terms of Riccati-type equations. In addition, the reduction technique is applied to stochastic systems for the first time and the usual restriction that system matrix is invertible is removed such that this technique is valid for some systems with singular system matrices; Thirdly, finite-horizon stochastic LQR problems are investigated for multiple input-delay systems and state-delay systems, respectively. Different from existing works which assume that the control weight matrix in the cost func-tion is positive definite to ensure that the optimal control is unique, this paper only requires that the control weight is positive semi-definite. In this case, a necessary and sufficient condition for the LQR problem admitting a unique solution is given.The main contents and results are listed as follows in the order of chapters:1. Complete solutions to the LQR problem and the stabilization problem for discrete-time stochastic systems with a single input delay are presented. A neces-sary and sufficient condition for the finite-horizon LQR problem having a unique solution is given. The optimal controller is shown to be a predictor form. Riccati-ZXL difference equations are established to characterize the optimal controller and the optimal cost function. A necessary and sufficient stabilization condition is pro-posed. In addition, our approaches to establishing relations between the costate and the state and to constructing Lyapunov functions based on the optimal cost supply new ideas to other LQR problems and stabilization problems for stochastic systems with delays, respectively.2. The stabilization problem for a class of discrete-time systems with multi-plicative noises and multiple input delays in the control variable is considered. Us-ing the reduction technique, the original system is transformed to a new stochastic system containing one input term. Motivated by the approach adopted to solve the stabilization problem for discrete-time stochastic systems with a single input de-lay in the first part, the stabilization condition for the reduced system is derived by considering its finite-horizon LQR problem first and then letting the horizon length converge to infinity. The novelty is as follows. A necessary and sufficient stabiliza-tion condition for stochastic systems with multiple input delays is established. The reduction technique is generalized to stochastic systems with nonsingular system matrices.For this new system, we solve its LQR problem in terms of a Riccate-type d-ifference equation. By showing the equivalence between the convergence of the solution to the difference equation and the stabilization of the original system, we develop a necessary and sufficient stabilization condition for the system under con-sideration.3. Finite-horizon LQR problems are investigated for two classes of discrete-time stochastic systems:one is with multiple input delays and another is with state delays, respectively. Our approach is to find relations between the costate and the state. The main contribution is as follows. A necessary and sufficient condition for the problem having a unique optimal controller is proposed. The optimal feedback controller and the optimal cost are presented via coupled difference equations.