A Study On The Relationship Between Distractor Types And Distractor Selection Patterns In An EFL Listening Comprehension Test

It is believed that miscomprehension underlies the selection of distractors in an English as a foreign language(EFL)listening comprehension test,so that the distractor selection behavior of EFL learners provides useful information for understanding their listening problems.This is contingent on the distractors having sound psychometric quality and the selection behavior having an observable relationship with listening ability.On this premise,the current study aims to investigate the psychometric quality of four types of distractors in the multiple-choice items of an EFL listening comprehension test and the relationship between distractor selection and listening ability.Accordingly,two research questions are put forward:1.Are there significant differences in psychometric quality among different types of distractors in a multiple-choice EFL listening comprehension test?2.How does listening ability affect the test takers’ selection of distractors?To answer question 1,three coders classified the 60 distractors of 20multiple-choice listening items in an experimental form of the College English Test Band 4(CET-4)into four types according to the nature of miscomprehension involved:complete,partial,confused,and derived.An item analysis was conducted on the response data of 3,318 test takers from 11 universities in China,followed by a frequency comparison of nonfunctioning distractors between the four types of distractors,a one-way ANOVA on the mean discrimination across distractor types,and a one-sample chi-square test to compare the observed proportion of each type of distractor to its expected proportion.To answer question 2,the test takers were divided into the low-,middle-and high-level groups according to their total scores in the listening comprehension test.To explore the relationship between listening ability and distractor selection,a chi-square independence test was then conducted between group and distractor type.The results indicated that in terms of distractor quality,the majority of distractors functioned well in the EFL listening comprehension test used in this study,and the few nonfunctioning distractors were evenly distributed among the four types.There was no significant difference in mean discrimination between the four types of distractors.However,the distractors reflecting complete miscomprehension were selected less than expected,whereas the distractors reflecting confused and partial miscomprehension were selected more than expectation.In terms of the absolute difference between the observed and expected percentages of selection by both distractor type and ability level,four of the differences were of practical significance,defined as a percentage difference greater than 2%.Specifically,the test takers of low listening ability tended to be attracted significantly more than expected by the distractors reflecting complete miscomprehension,and less than expected by the distractors reflecting partial miscomprehension.On the contrary,the high-level test takers were attracted significantly less than expected by the distractors reflecting complete miscomprehension,but more than expected by the distractors reflecting partial miscomprehension.As for the middle-level test takers,no distractor type was significantly more attractive than other types.The findings of this study contribute to a better understanding of the nature of distractors in EFL listening comprehension tests and how distractor selection behavior is associated to listening ability.They provide the empirical evidence in support of extracting information from distractor response patterns for diagnostic feedback and partial credit scoring.The findings also have technical implications for distractor development in multiple-choice EFL listening comprehension tests.