Psychometric analysis of student performance on basic medical science tests

The purpose of this study was to develop a framework for psychometric analysis of test scores that can be used with local testing in the context of medical education. Test scores from a basic science examination administered at a medical college during five successive academic years were analyzed in order to address questions about (a) psychometric characteristics of the test, (b) stability of student performance across academic years, (c) dependability of classification decisions (e.g. pass/fail), and (d) comparability of test scores across academic years.;The research questions in this study were addressed by using psychometric procedures in the framework of both classical test theory (CTT) and item response theory (IRT), as well as procedures bridging CTT and IRT in estimating true-score parameters of items and persons. The readily interpretable CTT information was combined with the more precise, yet more difficult to understand and interpret, IRT information to improve the quality of test score analysis.;The results of the study suggest that the test of interest has good psychometric characteristics. This test is a reliable instrument that can be dependably used for pass/fail classification decisions at the cutting score of 70 percent correct. Its measurement precision can be improved by adding more items that match the proficiency of the high-ability students. The findings of the study confirmed the expected stability of student proficiency across academic years but indicated that the current practice of measuring student performance through the use of CTT domain scores does not provide an adequate base for comparing students' performance across academic years.;This study can be used as a basis for developing a framework for psychometric analysis that will extend the boundaries of the current assessment practice and will provide more accurate and complete information for valid interpretations of test results and better informed decision making.