| Writing is of great importance in SLA teaching and ESL testing, while as to the rating criteria, most of the studies at home and abroad focus on the large-scale tests, few on achievement tests, which will reflect the problems of students and influence the teaching. The research is to probe into this field both quantitatively and qualitatively, with an aim to clarify the rating process and rating criteria in achievement tests. It is also hoped that the problems with the present rating criteria can be found out to maximize rating reliability and validity and to improve the diagnostic, pedagogic washback effects of writing tests.The thesis first reviews the rating criteria from two aspects: testing methods and scoring methods, and illustrates the criteria in tests and by researchers. Holistic scoring has the maximum construct validity, and can reflect the student's writing ability; its high rater reliability and emphasis on the writers' strengths are also its advantages, making it the most popular of all scoring systems. To achieve high rating reliability and validity, we need to establish suitable rating criteria to verify it, while the present rating criteria for achievement tests are mostly copied from large-scale tests, which are not suitable in content validity, authenticity and practicality.10 writing samples of the junior college students in the final exam were randomly selected to be scored by 10 teachers in Xinzhou Teachers University without any criteria. They were required to list three reasons and write down the retrospections of the rating process. The scores were analyzed by SPSS; the rating reasons were re-categorized with reference to Ling Shi's (2001) coding scheme; the retrospections of the rating process were analyzed qualitatively. The rating criteria achieved were testified in the re-rating stage, and the final scores were also analyzed quantitatively.No significant differences in the scores were found in reliability coefficients and one-sample T tests, and this means that the scores assigned by the 10 raters were close, while discrimination of the essays also existed. By analyzing the rating process, the author found that all the 10 raters used the similar rating criteria, with scores assigned through content, structure and language accuracy; while there was also subjectivity. Their understanding of the language ability in writing differed and rater training should be carried out before rating to maximize reliability and validity. In the rating criteria for achievement tests we should consider the requirement of language ability in the curriculums, and this is different from the rating criteria in large-scale tests due to the varied test types and test content. |
