Structural adaptive dynamic programming for a stochastic multidimensional aging and replenishment problem

Scalar aging and replenishment problems can be solved to optimality using classical dynamic programming techniques. Our aim is to study the structure of these problems and provide dynamic solutions that are scalable to the multidimensional setting.; We define the general class of aging and replenishment problems but we concentrate on the multiproduct batch dispatch problem. The multiproduct dispatch problem consists of heterogeneous products stochastically arriving at a station waiting to be dispatched by a finite capacity vehicle.; We begin our study with the scalar single product problem where we prove theoretical results on the structure of the value functions, and we characterize the optimal dispatch policies. The scalar problem provides a simple framework for developing and evaluating our solution methods. The main tool that we apply to tackle multidimensional problems is the use of structure in adaptive dynamic programming algorithms. We introduce structure in forward dynamic programming algorithms and demonstrate improvement in time performance. We then proceed to develop functional value function approximation algorithms that are scalable to the multidimensional case; we use linear and nonlinear functional approximations.; We proceed to the multidimensional multiproduct dispatch problem where we prove structural results on the value functions and the optimal dispatch policies. Then we extend the functional adaptive dynamic programming algorithms to the multidimensional case and show experimentally that they outperform their competition, which is a carefully tuned myopic heuristic of the sort that is often used in engineering practice.