Construct validation of an integrated-approach EAP placement test using multi-group structural equation modeling

This study concerns the construct validity of the video-reading based academic essay used as a part of the UIUC English as a Second Language Placement Test (EPT). The test emulates natural language behavior (e.g., lectures) by providing series of language modes such as listening and reading test inputs, each of which contributes to the overall context of language use. Thus, the present study intended to take the form of construct validation of the EPT in particular, and the EAP placement test developed using an integrated approach in general. There were two main research questions to clarify the construct definition of the test and the appropriateness of the task characteristics with respect to the construct definition: (1) to examine the unitary trait hypothesis of ESL proficiency related to listening and reading ability measured in the test; (2) to clarify the effects of listening, reading comprehension on the quality of ESL writing for both whole test sample and a set of subgroups of interest (nonpass and pass groups).;The research questions were tested using multi-trait multi-method (MTMM) procedures and multi-group structural equation modeling (MSEM) as the central statistical tool. A review of the statistical methodology of the MTMM procedures as a methodological contribution in literature preceded an investigation of the construct validity of the EPT. This powerful modeling technique enabled me to investigate comprehensively how listening and reading comprehension as proposed factors influenced or interacted with writing performance. Further, a fundamental concern in testing research, ensuring construct comparability when testing for between-group differences, was successfully carried out within the multi-group structural modeling framework. The findings were that while the effect of understanding the input is identical on the nonpass and pass groups, the two groups consistently differ from each other in three latent factors. Based on the findings of the EPT construct validation study, several recommendations from different perspectives of testing-related systems (test, rating, and placement system-related) are suggested for the future operation of the EPT program. Finally, methodological implications are presented.