The accuracy of unidimensional measurement models in the presence of deviations from the underlying assumptions

Scientific progress is dependent upon the discovery of unidimensional constructs and their interval measurement scales. Measurement models such as Classical Test Theory (CTT) and Item Response Theory (IRT) are commonly used in the human sciences to transform test or survey results from raw observations into scale scores that may or may not be interval.; It has been demonstrated that the one-parameter logistic (1PL), IRT model (also known as the Rasch model) is capable of producing interval scale estimates when the assumptions of unidimensionality, local independence, equal normal error, and no guessing are met. Estimates acquired using the normalized CTT model or the two-parameter logistic (2PL) or three-parameter logistic (3PL) IRT models can only produce approximately interval scale measures when their assumptions or conditions are met. However, the theoretical advantage of the 1PL/Rasch model is dependent upon assumptions that are unlikely to be perfectly realized in empirical data and thus the relative accuracies of results across models is unknown.; This study uses a series of nine simulations to examine and compare the accuracy of the item difficulty and person ability estimates produced by the CTT, 1PL/Rasch, 2PL and 3PL models when the data includes empirically justified deviations from the model assumptions.; The accuracy of person ability estimates were found to be comparable across models through each of the simulated conditions.; The accuracy of item difficulty estimates across the models were found to be dependent upon the information available (targeting), the degree of independent multidimensionality (variable discrimination) and the potential for guessing (pseudo-guessing). The 1PL/Rasch model produced the most accurate difficulty estimates only when the all of its assumptions were met. The CTT model results for item difficulty were similar to the 1PL/Rasch results in all simulations. The 2PL model produced the most accurate results when independent multidimensionality and mild guessing effects were added to the simulated data. The 3PL model produced the most accurate results when guessing effects were consistent with multiple-choice questions.; A visual conjecture describing the relative accuracies of difficulty estimates across the IRT models in a space defined by information, dimensionality and guessing is presented...