Evaluating risk prediction markers and models

Accurately predicting the risk of having an event, i.e. disease, is often the first step in medical decision making to ensure optimal preventive intervention and treatment. Rigorous statistical assessment of the performance of risk prediction markers and models is required in order to use them in clinical practice. For markers measured on a continuous scale, the distribution of population risks calculated using these markers is a useful tool. It provides a complete picture of marker performance and a meaningful way to compare the performance of different markers.;In this thesis, we describe measures to summarize and compare the predictive capacity of markers, all of which can be derived from the population distribution of risk. We propose some new clinically motivated summary measures and also give new interpretations to some existing statistical measures. Estimation of these measures under case-control and cohort study designs are discussed. We also develop distribution theory for these measures which can facilitate construction of confidence intervals from data. Asymptotic theory shows the estimators are asymptotically unbiased and normally distributed. Extensive simulation studies were performed to investigate their performance in finite sample studies. We illustrate our methods using cystic fibrosis data.;We next focus on a more specific question, estimating the increment in risk prediction with a novel marker. The diagnostic likelihood ratio can be used to quantify the change in risks obtained with knowledge of the new marker. And it can easily accommodate scenarios when baseline risk factors for the outcome exist. Methods for estimating the diagnostic likelihood ratio function is described along with the distribution theory. We use a neonatal hearing screening study to determine if the predictive information in a hearing test varies with baseline covariates. In addition, we illustrate using renal artery stenosis data how to estimate the covariate-specific predictiveness curve which is especially useful to guide individual decisions about marker ascertainment.;Finally for a continuous marker, methods to estimate the diagnostic likelihood ratio and the slope of the ROC curve are described. Because both the diagnostic likelihood ratio and the slope of the ROC curve are independent of the disease prevalence in the population, our estimation methods can also be applied to case-control designs. We perform simulation studies to investigate the performance of estimators. And these methods are further applied to two studies: one is a cystic fibrosis study and the other is a pancreatic cancer study. We also use the cystic fibrosis example to compare the predictive performance of two markers using their DLR functions.