Risk -sensitive control of discrete -time partially observed Markov decision processes

In this dissertation, we consider the following three problems arising in partially observed Markov decision processes: ergodic control, risk-sensitive average cost control, and adaptive control. First, we address the ergodic control problem. Our approach is intended to weaken the positivity assumption on the state transition kernel for every control action. Using the vanishing discount approach, we show that the dynamic programming inequality holds, and a stationary optimal strategy for Bellman's optimality is derived. Moreover, a benchmark partial replacement example is employed to illustrate that our assumptions are significantly less restrictive than others currently available.;Next, we consider the risk-sensitive control problem. The risk-sensitive control approach seeks to find an optimal policy to minimize a criterion that not only depends on the expected cost incurred, but also on the higher order moments. In fact, this approach is more conservative than the traditional risk-neutral control approach. We extend our ergodic control result to this problem. It is shown that there exists a bounded solution for the risk-sensitive average cost optimality equation, with a constant value of the optimal risk-sensitive average cost.;When some parameters of the Markov process are not available to decision- makers, we propose an adaptive control scheme to cope with it. The proposed adaptive policy is simple, recursive and easily implemented with no prior knowledge of the actual values of Markov decision process' parameters. We show that the proposed policy is self-optimizing in the long-run risk-sensitive sense when the sequence of parameter estimates converges to the true parameters, in probability.;Finally, we apply our risk-sensitive control scheme to a transmission scheduling problem in which a base station communicates with multiple remote users on a noisy wireless link. It is shown that there exists a stationary optimal policy for the problem under the risk-sensitive average cost criterion.