Examining the suitability of an IRT-based testlet approach to the construction and analysis of passage-based item sets in an EFL reading comprehension test in the Korean high school context

A testlet approach is currently being widely used as a method of test construction and analysis in many different contexts of educational measurement. The testlet approach involves using a set of related items as the basic unit of scoring and analysis by treating a group of items as a single polytomous item.;This study attempts to examine the suitability of an IRT (item response theory)-based testlet approach to the analysis and construction of passage-based item sets in the context of EFL (English as a Foreign Language) reading comprehension measurement. A 40-item EFL reading comprehension test was developed for Korean high school students, in which 4 different item types were used across 10 different passages. Five different forms of the same test were given to Korean 10th and 11th graders at 5 different high schools in South Korea.;The results of data analysis revealed that (1) a significant level of local dependence existed among items within passages in the EFL reading comprehension test; (2) item types made a significant difference in the proportion correct (p) index of items; and (3) score categories were ordered, even when item scores were totaled into a passage score. These findings provide support for using a testlet approach to the analysis of passage-based item sets and suggest that polytomous IRT models for ordered responses can be used to analyze the passage-based testlets along with the nominal response models.;The findings have three major implications for EFL reading comprehension measurement: (1) the models of test construction and analysis should be matched when passage-based item sets are used; (2) different subskills of reading comprehension may constitute a unidimensional construct measured by the item response theory; (3) more research is needed to determine the possibility of using the hierarchy of reading subskills in constructing a superitem in EFL reading comprehension tests.