Generalized monotone functional mixed models with application to modeling normal tissue complications

Normal tissue complications are a common side effect of radiation therapy. Examples include rectal failure in colon cancer patients and Xerostomia (loss of saliva production) in head and neck cancer patients. These complications are the result of the dose of radiation received by the normal tissue. While the tumor site generally receives a nearly uniform dose, the normal tissue does not. It is possible to obtain good estimates of the dose distribution to a normal tissue.; In chapter 2 we propose a Generalized Monotone Functional Mixed Model which relates the dose distribution to the expected probability or severity of complication. The dose effect is summarized as d p(d) * w(d) dd, where p(d) denotes the density associated with the dose distribution to the normal tissue. Within this model framework we propose a new method for nonparametric estimation of w(d) subject to two constraints. The first is that any biologically meaningful estimate of w(d) should be monotone. The second is that w(0) = 0. We define w(d) as the integral of a smooth positive function. The smooth positive function is obtained as a positive transformation of an unconstrained regression spline.; In chapter 3 we take a Bayesian approach to estimating w(d) within the same Generalized Monotone Functional Mixed Model. We define w(d) as a sum of monotone basis functions each multiplied by a non-negative coefficient. We specify a prior for these coefficients defined as a mixture of point mass at zero and a Gamma density. This prior enforces the monotonicity constraint and explicitly allows for flat regions in w(d) which correspond to regions of no dose effect. Gibbs sampling is proposed to estimate the parameters in this model, and we derive the full conditional distributions.; In chapter 4 we extend the model of chapter 3 to longitudinal complication data by allowing the weighting function w(d) to change over time. Several longitudinal models are discussed which impose varying levels of structure on the time varying dose effect parameters. All of the proposed methods are illustrated using a head and neck cancer dataset from the University of Michigan. The complication in this case is loss of saliva flow resulting from damage to the subject's Parotid glands.