| This dissertation examines the use of simulation as a tool for generating solutions to large-scale Markov decision processes (MDPs). In particular, it investigates the development of a simulation-based approach to generating aggregated MDP models which we call the Simulation for Model Generation (SMG) algorithm. Theoretical considerations of this approach are discussed, including the convergence behavior of the algorithm, as are issues surrounding the potential loss of the Markov property due to state space aggregation. The simulation methodology is then applied to two different application problems: a telephone network routing example and an inventory control example. An analysis of each of these problems is provided, and a series of exploratory tests are used to examine the behavior of the SMG algorithm in each case. Results of this preliminary testing demonstrate that simulation is able to capture the underlying behavior of a given system and that it can help produce very effective policy solutions even in situations in which very little specific problem information is provided. |
