Evaluating the effectiveness of two-stage testing for English and French examinees on the SAIP Science 1996 and 1999 tests

Two-stage testing (TST) is an adaptive testing procedure where test forms of varying difficulty are administered to examinees based on performance from a routing test. The School Achievement Indicators Program (SAIP) is the national achievement test in Canada and uses the TST procedure to assess educational progress of 13- and 16-year-olds in Science. SAIP works with the implicit assumption that the routing test works equally well for examinees in English and French. If this assumption is true, then there should be proper placement of English- and French-speaking examinees in the second-stage test. However, if the assumption is not true, then there might be misplacement of English- and French-speaking examinees in the second-stage test.;The purpose of the present study was to evaluate the effectiveness of a TST procedure for English- and French-speaking examinees who wrote the SAIP Science tests. The study was conducted using existing data (N = 24,642 and 22,320 for the Science 1996 and 1999 administrations, respectively) obtained from SAIP. The analyses were conducted in two steps. First, a comprehensive analysis of the routing test items was conducted using statistical and substantive methods. The purpose of these analyses was to identify items that might favor English- or French-speaking examinees, which in turn, might lead to different placement of examinees in the second-stage test. Second, a comprehensive analysis of the second-stage tests was conducted using BILOG-MG (Zimowski, Muraki, Mislevy, Bock, 1996). To assess performance differences for English- and French-speaking examinees on the second-stage tests, test information functions (TIF), test characteristic curves (TCCs), standard errors of estimate SE( q&d4; ), and reliability indices for English and French versions of the second-stage tests were compared.;Statistical analysis of the routing test items revealed that three out of twelve items displayed DIF. However, substantive analysis of the routing test suggested that translation errors were not the cause of DIF for the three items. Analysis of the second-stage tests indicated that English- and French-speaking examinees within low- and high-ability groups performed equally well in the second-stage tests suggesting that the routing test properly placed examinees in the second-stage test for both English- and French-speaking examinees.