An Empirical And Comparative Study On Holistic And Analytic Writing Scorings

As one important form of performance tests, writing has been attached great importance in recent years. It is widely used in large-scale English examinations, as well as in small-scale ones. Writing is featured by high validity, however, it lacks reliability in scoring. Because writing is a kind of subjective test, the answers are open-ended and thus provide discrepancy between different raters.No matter what the test method is, the reliability of ratings is one of the major issues in performance-based assessment. The evaluation and assessment of second-language writing has been one major area in testing writing, because there exist great controversies in the reliability of two scoring methods - holistic scoring and analytic scoring, but empirical studies on it are comparatively rare.This paper examines the strengths and weaknesses of holistic and analytic scoring methods, emphasizing on the comparison of the scoring efficiency, scoring reliability and testing decision of the two scoring methods. With the application of generalizability theory and specially developed program GENOVA, SPSS 13.0, the research has been carried out with 60 writing samples and 4 raters.The whole process of study can be divided into three steps: Firstly, 60 writing papers were collected, randomly selected and ordered; Secondly, four raters were asked to mark the 60 writing papers in the same period of time; Thirdly, they finished the questionnaire in a language lab.The analysis of the results prove at last: (1) Comparison shows that the holistic scoring is far less time consuming than the analytic one,and possesses significant advantage in economy and practicality; (2) The reliability coefficients– generalizability coefficients and Dependability coefficients– derived from holistic scoring are significantly higher than those of analytic scoring; (3) Due to its higher reliability, the conclusion of this study is that holistic scoring is more appropriate in high-stakes assessment program, if detailed information of candidates'writing ability is not required.These results indicate how we can apply the two scoring methods to better assess writing in tests. The value of this research lies in 2 aspects: (1) Extension of the findings of Bachman and McNamara. (2) Suggestions to advance writing scoring administration and contribution to the current discussion about test fairness.In the end, the paper shows research drawbacks and areas that more attentions are to be given in future scoring methods research.