Construct validation of a diagnostic L2 listening test: An operational model utilized and multidimensionality issues revisited

The primary purpose of this study is to develop and validate a diagnostic test for measuring English as a Second Language (ESL) students' listening comprehension ability. This study began by exploring a parsimonious model of listening. A model was proposed as a basis for operationalizing the construct of listening performance. The model contains four major tasks, and each task consists of two salient traits: process type (global and local) and input range (wide and narrow). The proposed model and test design were applied to actual diagnostic assessment from University of Illinois ESL service program in the spring semester of 1999.;An argumentative model by which to judge construct validity evidence was proposed. Construct validity evidence was mainly accumulated to determine the extent to which the diagnostic test represents theorized task types. Four types of construct validity evidence argued for and against inferences that variance in test response can be attributed to lecture comprehension ability: content evidence, empirical item analysis, correlational evidence, and internal structure.;After analyses of test results, a validity table was presented to argue both for and the construct validity of the diagnostic listening test. The main argument from these results is that the diagnostic listening test did not successfully measure what it was designed to measure. However, the strong relationship between test performance and dichotomous traits of comprehension input range across all data analytical stages provided a partial justification for the construct validity of the listening test. This study offers positive expectations on the basis of item input range for achieving both a multidimensional and a valid test of ESL listening, but further investigation of the target test construct is needed.;The present study was exploratory to bring a new perspective to multidimensionality issues in language testing. It encouraged testing experts to revisit the issues of multidimensionality from the perspective of test users' views. It argues for the meaningfulness of test users' needs and wants to raise a potential shift toward post-psychometric approach in the development of multidimensional language tests. This study can be placed in the broad project of exploring multidimensional nature of language tests.