A Dissertation Submitted In Partial Fulfillment Of The Requirement For The Degree Of Doctor Of Philosophy In Applied Linguistics

The pursuit for quality and professionalism in language testing and assessment has become an apparent trend in the past decade, as is evidenced by a number of standards or codes of practice which have been developed, implemented or enforced by testing organisations from all over the world (see e.g., AERA/APA/NCME, 1999; ALTE, 1994; EALTA, 2006; ETS, 2002; ILTA, 2000, 2007; JLTA, 2003). Concomitantly, due to the open-door policy and the surging number of English learners, EFL testing has been developing with great momentum in China. These EFL tests are normally large-scale and standardised, and the results on these EFL tests are often used to make high-stakes decisions which have major impact on stakeholders, institutions and society as a whole. In addition, due to the prominent role that tests and examinations play in China's society, these tests often have strong washback effects on English teaching and learning, and a heavy social weighting.This study is intended to develop a code of practice for the EFL tests in China (hereinafter the Code) with respect to test development, administration and use. Different from many other initiatives to develop language testing standards which are based mainly on the drafter's or developer's subjective or experiential understanding about the contextual features, this study adopts a data-based approach, which means both the structure and the guidelines of the Code are primarily determined by the data collected in this study. To fulfil the purpose of developing the Code, this study is intended to address the three research questions which are listed in the following:1) What are the language testing standards which have already been developed, implemented or enforced in other testing contexts, and what are the implications of these initiatives of developing and enforcing standards to the present study?2) What are the features of the context in which the Code is to be developed and put into use?3) What are the structure and guidelines of the Code, and how is the structure and guidelines informed by the data we have collected? To adequately clarify these three research questions, this study adopted a three-stage design. At Stage 1, major standards in language testing were collected, reviewed and critiqued. These standards were developed by testing organisations from all over the world, and were recognised as having important impact on the international language testing community, including, for example, the ILTA Code of Ethics (ILTA, 2000), Standards for Educational and Psychological Testing (AERA/APA/NCME, 1999) and ETS Standards for Quality and Fairness (ETS, 2002). The review of the standards helped to establish a preliminary model which demonstrated the process and fundamental considerations in developing standards in language testing.At Stage 2, an empirical investigation was conducted with a view to portraying a representative picture of the current EFL testing situation in China. The participants in the study were six EFL examination boards, 490 students and 166 teachers from the tertiary education in China. The primary research instruments that were employed in data collection were three questionnaires, administered to the examination boards, students and teachers respectively. In addition, three checklists and an interview guide were also used in this study. The survey of the examination boards identified the strengths, and more importantly, the areas which were recognised as in need of improvement in the EFL testing practices. Issues such as pretesting, communicating test results and test use were identified as particularly problematic, which might pose threat to test validity and fairness. The investigation of students'and teachers'perceptions of the EFL testing practices lent further support to the findings in the survey of the examination boards. Students and teachers on the whole had positive perceptions of the EFL tests. The negative perceptions included the design of some EFL tests which were believed to be heavily reliant on the MCQ format, the accuracy of marking, cheating and plagiarism, and the inadequate assessment of spoken English abilities, etc. The empirical investigation at this stage heightened the necessity and urgency of developing a code of practice for the development, administration and use of these EFL tests. At Stage 3, the Code was drafted on the basis of the studies at the previous two stages. Before drafting the Code, some fundamental considerations in developing language testing standards were revisited. The investigation at Stage 2 suggested that the audience of the Code should not be confined to EFL examination boards but be extended to stakeholder groups such as the educational authorities, administrators, EFL learners and educators, etc. Test validity and fairness were believed to be the joint responsibility of all stakeholders in the testing process. In addition, it was believed not practicable to prescribe the enforcement mechanisms in the Code, and the purposes of the Code, at the present stage, were primarily educational and inspirational, or in more detail, were two-fold: to raise the awareness of professionalism and quality among the EFL test developers, and to communicate to the stakeholder groups the basics of language testing and good testing practice.This study, the first research effort ever to develop a code of practice for China's EFL tests, is believed to represent a small yet momentous step in pursuing even better quality and professionalism in EFL testing. The Code itself, as well as the process of developing the Code, we believe, has important implications for EFL testing theory and practice in the Chinese context. The shift proposed in this study from a code serving educational and inspirational purposes to a code which is used to guide and evaluate local testing practices with applicable enforcement mechanisms provides the test developers in China with a practicable approach to pursuing better quality and professionalism through developing their internal codes. The bulk of this study is concentrated on the a priori validation of the Code. In the future, however, there is much more validation work to do which is beyond the scope of this thesis, such as investigating stakeholders'views and comments as to the content, appropriateness and applicability of the Code, and examining the consequences and impact, if any, of the Code.