Optimal Robust Decision Problems For Uncertain Semi-Markov Systems

Semi-Markov decision processes (SMDPs) can be used to describe a large class of discrete events dynamic systems in the real world, namely semi-Markov systems. The uncertainty inevitably exists in real systems, and the optimization of systems often involves multi-extremum problems. Therefore the study of robust decision for uncertain SMDPs is more practical and important. The main work of this thesis is to research the application of global optimization methods to derive the optimal robust control policy of uncertain SMDPs.In general, the analysis and research of SMDPs are performed through equivalent Markov decision processes (MDPs), and the research on uncertain SMDPs is based on the work about certain SMDPs. So we firstly introduce equivalent MDPs and some performance potential based optimization methods for certain SMDPs. Then, we introduce some performance potential based algorithms for the robust decision of uncertain SMDPs, which can be unified and used for both average and discounted criteria. To solve the multi-extremum problems involved in the solution of robust control policy, we especially introduce two different types of global optimization methods, that is, simulated annealing and filled function approaches. We discuss in detail the application of global optimization based methods in the robust decision problems for various uncertain SMDPs, including policy iteration algorithm for independent uncertain parameter case and a class of min-max optimizing algorithms for dependent parameter case. There are different conditions in these two cases. In the former case, only the equivalent infinitesimal generator relies on uncertain system parameters, and in the later case, both the performance function and the equivalent infinitesimal generator rely on uncertain system parameters.With numerical experiments on solving robust decision problems of SMDPs, we explain the application of different algorithms in various uncertain cases. The examples show that these algorithms are uniform for both average and discounted criteria. By analyzing a large amount of experiment results, we discuss the optimizing effect of various algorithms in different cases, and give some suggestions on how to select and use the algorithms for different cases. The research results of this thesis have certain theoretical and practical value in establishing robust decision mechanism of some real semi-Markov systems.