An Empirical Study Of Authenticity In The Assessment Of EFL Writing Test

Authenticity has been recognized as a crucial issue in language testing. Authentic language testing is perceived as a real indicator of students' language proficiency and ability of applying language. To design an authentic test, the principle of authenticity should be considered in the whole process of language testing, such as the choice of task, the form of testing materials, the pattern of the items, the implementation, the rating procedure and the interpretation of the scores. Scoring is an aspect of authenticity that has not been discussed as widely as it should be. There remains blank in the field of authentic-inauthentic rating procedure of writing test either domestically or abroad.The research designed in the thesis is to investigate the authenticity issue in the rating procedure of subjective EFL writing test. Specifically, the quantitative analysis concerns the following questions:1) Whether the teachers can adopt different rating criteria, thus can be classified into two groups, i.e. tradition-based rater (TBR) and authenticity-based rater (ABR).2) If they can be distinguished, whether difference of scores from two groups of raters exist significantly.3) Whether low score group and high score group classified by tradition rating criteria have significant difference in their scores rated by two teacher groups.Qualitative analysis discusses whether two groups of teachers have different interpretations on the scores.The subjects of the experiment are 20 English teachers. They are classified into two groups according to Bachman and Palmer's (1996) theory of authenticity, and each group contains 10 teachers respectively. In order to distinguish these two groups scientifically, chi-square test is used to provide an empirical evidence for the definition of the two groups of teachers. Twelve writing samples were selected systematically from a writing class of second-year students in the school of foreign language of Shanxi University. 20 teachers were asked to rate the 12 writing samples separately, and state three reasons or comments, in the order of importance, for their ratings of each essay in their questionnaires.The results of present study demonstrate that scores showed no significant difference between ABR and TBR raters. However, as to high-score group, the difference of two sets of scores rated by two different rating criteria is not significant, but for low-score group, two sets of scores have significant divergence.The two groups of teachers differed markedly in the frequencies of certain types of comments or criteria they mentioned to justify their ratings. The ABR raters were more aware of the ability of language application in situation of authentic language use such as content relevance, intelligibility and task completion. In contrast, TBR raters were found to focus significantly on language writing knowledge such as grammar, syntax and organization.Finally, based on the findings, implications and limitations of the present study were discussed and further research directions were suggested.