| Model selection is often employed in many scientific studies and has a significant effect on inference of estimated parameters. One common practice in modeling is to select a single best model from a set of candidate models using one type of model selection method. After the model has been selected, one usually applies the standard inferential methods to an estimator directly as if the model had been known in advance. This process simply ignores the uncertainty introduced by model selection, and could yield overoptimistic inferences, resulting in misleading conclusions.; In this dissertation, we first discuss the issues of inference after model selection in the context of dose-response analysis. A general methodology (Shen et al. 2004) is reviewed for estimating the variance of the complex statistics that involve the process of model selection. Simulation results are used to demonstrate the performance of data perturbation in estimating the standard error of parameter estimates and the prediction error, as well as the impact of model selection on estimation. The application of this method to a phase I clinical trial data is considered.; A simple example of linear regression is shown to highlight the difficulty of deriving the statistical properties of estimators after model selection, as well as the effects of the uncertainty of model selection. A data perturbation approach is proposed to obtain an estimate of model selection probability. The performance of this method and its application in hypothesis testing are demonstrated through simulation. We also proposed a method to construct a confidence interval through data perturbation. The performance, as compared to the common alternatives, is examined through simulation in various situations. |
