Assessing Model Fit of Multidimensional Item Response Theory and Diagnostic Classification Models using Limited-Information Statistics

Educational assessments have been constructed predominately to measure broad unidimensional constructs, limiting the amount of formative information gained from the assessments. This has led various stakeholders to call for increased application of multidimensional assessments that can be used diagnostically to address students' strengths and weaknesses. Multidimensional item response theory (MIRT) and diagnostic classification models (DCMs) have received considerable attention as statistical models that can address this call. However, assessment of model fit has posed an issue for these models as common full-information statistics fail to approximate the appropriate distribution for typical test lengths. This dissertation explored a recently proposed limited-information framework for full-information algorithms that alleviates issues presented by full-information fit statistics. Separate studies were conducted to investigate the limited-information fit statistics under MIRT models and DCMs.;The first study investigated the performance of a bivariate limited-information test statistic, termed M2, with MIRT models. This study particularly focused on the root mean square error of approximation (RMSEA) index computed from M2 that quantifies the degree of model misspecification. Simulations were used to examine the RMSEA under a variety of model misspecifications and conditions in order to provide practitioners empirical guidelines for interpreting the index. Results showed the RMSEA provides a useful indicator to evaluate degree of model fit, with cut-offs around .04 appearing to be reasonable guidelines for determining a moderate misspecification. However, cut-offs necessary to reject misspecified models showed some dependence on the type of misspecification.;The second study extended the M2 and RMSEA indices to the log-linear cognitive diagnostic model, a generalized DCM. Results showed that the M2 followed the appropriate theoretical chi-squared distribution and RMSEA appropriately distinguished between various degrees of misspecification. Discussions highlight how the limited-information framework provides practitioners a pragmatic set of tools for evaluating the fit of multidimensional assessments and how the framework can be used to guide development of future assessments. Limitations and future research to address these issues are also presented.