A Study On The Rating Scale Of National Matriculation English Writing Test

This thesis aims at making a study on the rating scale of National Matriculation English Writing Test(NMEWT). Writing assessment of EFL has been a major part in the field of language testing. National Matriculation English Test(NMET) is a large-scale and high-stakes test in China. However, its rating scale of writing test used nowadays is a holistic one constructed on the base of former linguistic competence theories and teaching syllabus, which brings some problems like ambiguous rating dimensions, lower rating validity etc. in its rating process. Recently, some studies have verified that analytic rating scale can provide more specific diagnostic information for English writing teaching, so as to evaluate examinees’ ability more accurately from different aspects. It can also improve the validity of writing assessment and the fairness of language testing. Therefore, this study intends to construct an analytic rating scale for NMEWT by using the qualitative and quantitative method to make an exploration in the innovation of the rating scale for NMEWT and guarantee the validity of NMET.The present study is based on Bachman’s(1996) Communicative Language Ability(CLA) and Weir’s(2005) Socio-cognitive Framework of Validating Writing Tests(SFVWT) to explore the following research questions: 1) What are the dimensions of the rating scale on NMEWT? 2) What is the weight of each dimension onthe rating scale?3) To what degree is the rating scale valid?To answer the above questions, the research is designed as following procedures: First of all, in order to decide the dimensions of the rating scale, the research methods of literature review, interview and questionnaire are employed. Based on Bachman’s CLA, the descriptors of the rating scale are collected by means of literature review from those well-known domestic and foreign rating scales like IELTS, TOEFL, CET-4, TEM-4 etc., and the interviews of English teachers in senior high schools and raters in NMET. After the collection of descriptors, the researcher designs a questionnaire to explore the importance that those descriptors play in the rating scale on NMEWT from the views of English teachers, raters and students, so as to find out the valid components of the rating scale. Exploratory factor analysis of collected data is used subsequently to investigate the main dimensions of the rating scale. Secondly, on the purpose of investigating each dimension’s weight, the researcher invites two experienced raters to score thirty writing samples on the task of NMEWT(2015) in Hunan Province. The step-by-step method in regression analysis is used to get the regression coefficients of different dimensions in this initial rating scale, so as to make an ultimate decision on the weight of each dimension on the rating scale. Thirdly, in order to verify the validation of this new rating scale, the researcher asks four raters to score thirty writing samples on the task of NMEWT(national version) in 2015 based on the new analytic rating scale and the old holistic one. The evidence collects from two aspects: the comparative study on scoring validity between two rating scales and the analysis of the analytic rating scale based on MFRM on their inter-rater consistency, intra-rater consistency, the interaction between raters and the rating scale and rating scale’s distinguish function.The results of this study are as follows: 1)four dimensions of the rating scale on NMEWT are: rhetorical organization, ideas and arguments, language use(accuracy, complexity and appropriateness) and written mechanics; 2) the weight of each dimension is: rhetorical organization 15%, ideas and arguments 15%, language use 52% and written mechanics 18%; 3) the rating validity of this analytic rating scale for NMEWT is favorable and it can effectively distinguish the writing ability of different examinees. The inter-rater consistency, intra-rater consistency, interaction between raters and rating scale and rating scale’s distinguish functionhave been obviously improved compared with the holistic rating scale. Therefore, the results indicate that it’s feasible to construct an analytic rating scale for NMEWT. The construction of a rating scale for English writing test should apply the method of theory-based and data-based. In the process of constructing a rating scale, collecting descriptors is the base, determining the dimensions and weights are the key point and validating is the significant guarantee.Hopefully, this study can provide reference for the innovation of the rating scale on NMEWT and the teaching and assessing of English writing in senior high schools.Due to limited time, energy and the constraint of objective environment, there are still some limitations in this study. At first, the writing samples used in this study are re-tested compositions. Influenced by examinees’ psychological factors, there still exist some difference between the re-test samples and the real compositions, though the author has tried her best to make them the same. Secondly, the number of experienced experts and raters of NMET involved in this study are limited, and in the future study, more experts should be involved to expand the sample and make the research more comprehensive. The interaction between raters and rating scale in NMEWT and the exploration on the rating process are worth deep investigating in further research.