Simultaneous optimal parameter selection and dynamic optimization using iterative dynamic programming

This thesis consists of two main parts. In the first part, two procedures for improving the convergence property of iterative dynamic programming (IDP) in yielding the global optimum are presented. In the second part, we focus on the extension of IDP to optimal control problems where some of the time-invariant parameters in the initial condition and/or in the state equation are not specified, and must be chosen as a part of the optimization problem.; To increase the chance in achieving the global optimal solution, in addition to randomly chosen candidates for control, we include deterministic control candidates into the search space of IDP. Two types of deterministic control candidates (shifting and smoothing candidates) are chosen based on the control policy obtained in the previous iteration. The search for the optimal control value in the subsequent iteration is then made on the combined set of control candidates chosen randomly and deterministically. Three nonlinear optimal control problems are chosen to illustrate and test the procedure.; To improve the convergence rate of IDP, we suggest the use of an adaptive scheme for region size determination. In this procedure, IDP is used in a multi-pass manner where the initial region over which control candidates are chosen for a subsequent pass is based on the extent of variation of the control variable in the current pass. This procedure, as illustrated and tested with two highly nonlinear chemical engineering problems, enables the optimum to be determined more efficiently as compared to the conventional scheme of restoring the region size to a fraction of the size used at the beginning of the previous pass.; When the initial condition is not rigid as in the case of a fed-batch reactor where the initial volume is quite arbitrary, optimization can also be applied to determine the “best” initial condition to use. To apply IDP to an optimal control problem in which the initial values of some of the state variables are flexible and can be chosen by a user, we suggest that the free initial condition be taken to be an additional control variable for the first time stage only. In this way, the search for the optimal initial condition and also the optimal control policy can be carried out simultaneously using IDP. This method is straightforward; and the computational experience with three nonlinear optimal control problems shows that this approach provides an efficient and reliable means for solving optimal control problems with some flexibility in the choice of the initial condition.; When the free time-invariant parameters are present in both the initial condition and the state equation, we propose to first transform all unspecified parameters in the state equation into free initial conditions through the introduction of some extra state variables. Then the procedure for solving optimal control problems with a free initial condition is used to simultaneously determine the optimal control policy and the optimal values for the free parameters. As illustrated and tested with three nonlinear optimal control problems with unspecified parameters in the state equation, the use of this transformation approach with the simultaneous search is found to work well even for a highly nonlinear system with five unknown time-invariant parameters.