Optimal Control For Stochastic Markov Jump Systems With Input Delay

The Markov jump system is a random system with multiple modes and the system's jump transition between each mode is determined by a set of Markov chains.Due to its characteristics that the system parameters can change abruptly in the representation process,it has been widely used in financial Economic,intelligent transportation,manufacturing and other engineering applications.However,in the process of data transmission,time delay or packet losses will inevitably occur because of insufficient network bandwidth,component failures,buffer overflows or other reasons.These phenomena will affect system performance and even damage system stability.In this paper,we mainly investigate the optimal control problem of Markov jump linear system with input delay.For stochastic system and Markov transition linear system with input delay or packet loss,based on the stochastic maximum principle and the second Lyapunov method,we present some design methods of the optimal tracking controller,the optimal state feedback controller and the optimal output feedback controller.Moreover,the stabilizing control is studied in the infinite-horizon.The specific research work is as follows:The tracking control for stochastic system with input delay or packet losses.For the stochastic linear system with input delay and the network control system with packet losses,we first establish an augmented system related to system equation and reference trajectory in finite-horizon.Based on the maximum principle,the optimal tracking controller is obtained by solving a set of difference Riccati equations.In infinite-horizon,a set of algebraic Riccati equations is introduced to obtain the sufficient and necessary condition for the existence of optimal tracking control,and an optimal tracking controller which can ensure the stability of the system is proposed.Finally,the validity of the conclusion is verified by MATLAB simulation analysis.It is worth mentioning that,we apply the maximum principle to design the optimal tracking controller for a stochastic system with input delay for the first time.For two types of system,although we establish an augmented system during the design process,the optimal tracking controller has the same dimensions as the original system,which effectively reduces the computational complexity.The optimal state feedback control of Markov jump linear system with input delay.For a Markov jump linear system with input delay and additive noise,we assume that the controller can obtain the system state variables information and the transition parameters information in finite-horizon case.According to the maximum principle of stochastic system,a new stochastic maximum principle which can be used for Markov jump linear system is introduced.And based on the maximum value principle,we obtain a sufficient and necessary condition for the existence of a unique solution to the optimal state feedback control problem by solving a set of coupled difference Riccati equations,an optimal state feedback controller is proposed in finite-horizon.Finally,we give numerical examples and use MATLAB for simulation analysis to verify our conclusions.It should be pointed out that we introduce a new stochastic maximum principle suitable for Markov jump linear systems based on the maximum principle of stochastic systems,and prove the correctness of this principle by using the calculus of variations.In addition,different from the dynamic programming method,we use the new stochastic maximum principle to design the optimal state feedback controller.The optimal state feedback controller obtained can effectively suppress the influence caused by time delay and meet the requirements of control performance.The optimal output feedback control for Markov jump linear systems with input delay.For Markov jump linear system with input delay and packet loss,we assume that the controller can obtain the jump parameters information,but unable to get system state variables information.Firstly,an optimal filter is designed to obtain the estimate of the state of the system by solving a set of coupled filter Riccati equations.Based on this filter,we propose an optimal output feedback controller by solving a set of coupled difference Riccati equations,and prove that the separation principle is also valid.Finally,we use MATLAB for simulation analysis to verify the correctness of the conclusions.We introduce an additive noise which may affect the performance index into the system equation,and the derived results can be applied to a wider range.Moreover,compared to the Riccati equations developed for Markov jump linear system without delay,the Riccati equations developed for Markov jump linear system with input delay involves delay terms,which is caused by the delayed backward difference equation in the maximum principle.The stabilizing control of Markov jump linear system with input delay in infinite-horizon.It is assumed that the controller can obtain system states information and Markov jump parameters information.According to the study of optimal state feedback control,we first present an optimal state feedback controller in finite-horizon.In infinite-horizon,an augmented system is established to transform the time-delay system into a time-free system.It is proved that the coupled difference Riccati equations defined in finite-horizon converge to a set of coupled algebraic Riccati equations when time tends to infinity.Next,the sufficient and necessary condition for the system's mean square stability are given and an optimal stabilizable controller for Markov jump linear systems with input delay is proposed.Finally,we give some numerical examples and use MATLAB simulation analysis to verify the effectiveness of the proposed controller.It should be pointed out that,in infinite-horizon,a sufficient and necessary condition is given for the existence of mean square stabilizing control for a Markov jumping linear system with input delay for the first time.