Construct validation of an integrated, process-oriented, and computerized English for academic purposes (EAP) placement test: A mixed method approach

The Computerized Enhanced ESL Placement Test (CEEPT) at the University of Illinois at Urbana-Champaign is a daylong process-oriented writing assessment in which test takers are given sufficient time to plan, produce, and revise an essay. This is a comprehensive validation study of the CEEPT, rooted in the modern paradigm of test validation. I employ Messick's (1989) validity framework to build my arguments and I use multiple types and sources of evidence. The research design is mixed in terms of sources and types of data. Data were gathered not only from test takers but also from faculty, and quantitative and qualitative forms of data gathering were employed.; A validity table is presented to summarize the positive and negative attributes of six research questions investigated in this study: predictive validity, convergent and discriminant validity, improved essay quality, consequential validity, test fairness, and authenticity. Arguments in favor of the CEEPT came from six types of validity evidence. First, the qualitative data complemented the results of a correlation coefficient between CEEPT and first semester GPA. Second, the CEEPT is convergent with the TOEFL listening, the TOEFL essay writing, and the GRE analytical writing sections. The CEEPT measures different abilities than do the quantitative and analytical sections of the GRE, as indicated by insignificant and low correlation coefficients. Third, the CEEPT elicited test takers' writing abilities with the help of peer feedback and computer writing tools, as indicated by scores and text analysis. Fourth, malcontents (i.e., test-takers unhappy with the test's results) reported that they benefited from taking ESL courses and appreciated the benefit, which showed that adverse consequences were minimized. Fifth, an absence of evidence of bias ensures comparable construct and predictive validity for subgroups of language background, gender, and discipline. Sixth, students perceived a close match between the academic tasks and the CEEPT tasks, and they expressed their preference for the CEEPT and the computer delivery of the test.