| This dissertation formulates and solves a combined estimation and optimal control problem for a finite-state, discrete-time, partially observed, controlled, hidden Markov model with unknown state transition and output transition matrices. The cardinality of the state and the cost structure are assumed known. The control implemented at each step is a randomized approximation to an optimal risk-sensitive control, and is calculated with the current value of the plant estimates. The degree of randomization is determined by the value of a positive parameter which is allowed to decay to zero at a constant rate. As the parameter decays to zero the control converges to an optimal control for a moving horizon risk sensitive criterion.; The main contribution of the dissertation is the presentation of a stochastic approximation proof for the asymptotic convergence of the algorithm for combined estimation and control. The proof requires the development of a potential theory for the Markov chain that captures the combined dynamics of the hidden Markov model, the estimator and the control algorithm. The potential kernel associated with this chain is shown to be regular with respect to variation in the plant estimates which influence the kernel both directly though the estimation algorithm, and indirectly through the control algorithm. |
