Robust Optimal Control for Medical Treatment Decisions

Medical treatment decisions, such as glycemic control for type 2 diabetes patients, involve complex tradeoffs between the risks and benefits of treatment. The diversity of treatment options that patients can choose over time and uncertainty in future health outcomes results in a difficult sequential decision making problem. In this dissertation, we focus on developing quantitative models and methods for medical decision making, and we apply these models in the context of treatment decisions of glycemic control for patients with type 2 diabetes. We begin by reviewing the background of type 2 diabetes with the focus on treatment options for glycemic control. We also provide a review of the Markov decision process (MDP) and robust optimization literature which is relevant to the methodological aspect of this dissertation. We present a glycosylated hemoglobin (HbA1c) Markov model, and use this model to evaluate and compare effectiveness and cost of treatment regimens for new and old hyperglycemia lowering medications for individuals newly diagnosed with type 2 diabetes. Then, we present an MDP, based on the HbA1c Markov model, to optimize the sequence and time to initiate cost-effective medications with the objective of maximizing a patient's quality-adjusted life-years prior to the first adverse event defined as a micro- and/or macro-vascular complication (e.g., heart attack, stroke, kidney failure) or death from any cause. The MDP optimally trades off the potential benefits from reducing HbA1c with the disutility associated with side effects of taking hyperglycemia lowering medications. We also analyze the impact of hypoglycemia on the MDP based optimal policy, and compare the MDP based optimal policy with published treatment guidelines. Finally, we extend the MDP to a robust MDP treatment model (RMDP-TM), which considers uncertainty in transition probability matrices. The RMDP-TM with a general uncertainty set is very difficult to solve. To deal with this, we relax the RMDP-TM and incorporate a new uncertainty set model for which we show that the relaxed RMDP-TM can be solved exactly. We also provide a fast algorithm to solve the RMDP-TM approximately. We present theoretical analysis related to properties of the RMDP-TM which can be used to achieve computational efficiency of solving the RMDP-TM. We conclude by summarizing the most important conclusions that can be drawn from this dissertation.