The effect of specificity of language test specifications on item construction

In criterion-referenced testing, test specifications presenting the explicit guidelines for the construction of test items have been regarded as a significant tool for securing content validity of the test. It is because the requirements or conditions for the development of test items stated in test specifications are believed to help construct the homogenous items originally intended, delimiting the scope of test items. The dilemma of this item construction method lies in the high degree of explicitness: The item writer is likely to ignore the detailed description of test specifications due to practical reasons such as lack of time and/or expense. Keeping these concerns in mind, this research aims to investigate the effect of the explicitness of language test specifications on item construction.; For the purposes of this study, three test specifications for an ESL reading comprehension test differing in their degree of explicitness were developed. On the basis of these three test specifications, nine item writers developed 54 items. A test consisting of 15 items chosen out of the 54 was administered to 527 ESL students.; The 54 items developed were examined by three judges with respect to the quality of items. The assessment of the judges showed that the items developed on the basis of less explicit test specifications were better than those using more explicit test specifications. The statistical analysis of the test using classical test theory, item response theory, and distracter analysis, however, showed somewhat different results in that the more explicit test specifications were better than the less explicit ones in the construction of correct response and distracters.; The results of this study led us to make the following suggestions: (a) test specifications do not have to be as explicit as they could be, (b) the construction of test specifications should reflect the item writer's opinions and difficulties, and (c) test development should be an ongoing process. In order to implement the aforementioned suggestions, more research oriented towards qualitative and longitudinal paradigms, which is expected to reveal test development processes, should be conducted in the language testing field.