An Empirical Study Of Source Use,linguistic Features And Writing Quality In Integrated Writing Tasks

As a growing number of high-stakes,large-scale tests use integrated tasks,more validation studies are needed to inform stakeholders about their construct and interpretation.The underlying reasons for using integrated tasks lie in their perceived authenticity and the background they provide for test takers.Different integrated writing tasks have been used in large-scale tests in China in recent years(e.g.,National College Entrance Examination,Test for English Majors,and National English Writing Contest).Based on the assessment use argument by Bachman and Palmer,the present study investigated the process of source use in integrated writing among second-year English major students.Furthermore,this study examined the relationship between source use,linguistic features and the writing quality in integrated writing tasks.In terms of research procedure,the present study conducted a survey to elicit EFL students’ perceptions of source use in the integrated writing process.The questionnaire items were adapted from relevant studies of reading-to-write strategies.The questionnaire was checked by 35 second-year English majors in the pilot study.The questionnaire’s content and language were revised based on their feedback.In the main study,questionnaires were distributed to 124 sophomore English majors at a university in southern China.In the end,111 valid questionnaires were collected.In addition,participants were given 45 minutes to complete the integrated writing task in the classroom.The batch of 124 essays were rated by two experienced teachers,with an interrater consistency of 0.863.For data analysis,exploratory factor analysis was conducted to check the underlying factors of the source use in the writing process using the principal components analysis extraction and varimax rotation.Three indicators of source use features were selected,including importance score(IS),implicit source use(ISU)and verbatim source use(VSU).Six indicators of linguistic features were selected,including clauses per T-unit(C/T),dependent clauses per clause(DC/C),corrected type-token ratio(CTTR),error-free T-units per T-unit(EFT/T),errors per T-unit(E/T)and number of words(W).Among them,four measures(C/T,DC/C,CTTR,W)were automatically coded with the L2 Syntactic Complexity Analyzer(L2SCA)and Lexical Complexity Analyzer(LCA),whereas source use features(IS,ISU,VSU)and linguistic features(EFT/T,E/T)were manually coded by two researchers.Correlation analysis and regression analysis were then carried out to examine the predictability of source use and linguistic features in TEM-4 integrated writing tasks.In factor analysis of the source use in the writing process,four underlying factors were identified: factor 1,gaining ideas;factor 2,integration process;factor 3,language support;and factor 4,opinion formation.Descriptive statistics and paired sample t-test suggested a general frequency pattern of the source use in the writing process.Overall,the reading-to-write task provided the research participants with main ideas,examples and language support.Comparatively,integration process such as paraphrasing and summarizing were less frequently used by the participants of the present study.A one-way analysis of variance(ANOVA)was conducted to investigate the nine indicators of source use and linguistic features across three proficiency levels.The results yielded significant differences across proficiency levels for two source use features(IS,ISU)and four linguistic features(CTTR,EFT/T,E/T,W).Regression analysis further revealed that source use and linguistic features had a cumulative impact on writing scores.Three indicators(the importance score,essay length and corrected type-token ratio)explained 49.8% of the variance in integrated writing scores.Among the three indicators,essay length had the greatest predictability and explained 42.2% of the variance in scores on the integrated task,followed by corrected type-token ratio(5.7%)and the importance score(1.9%).These findings make an important contribution to the validation of integrated writing tasks.The process of the source use in integrated writing tasks was revealed,and the predictability of source use and linguistic features in TEM-4 integrated writing was discussed.At the theoretical level,the present study provided empirical evidence of integrated score interpretation under the assessment use argument framework.At the methodological level,the present study tentatively combined source use and linguistic features in the prediction model,which has implications for researchers and developers of integrated tasks.At the practical level,the findings benefit potential test takers by providing information about the marked source use and linguistic features of high-score compositions.