Research On CATs That Allow Revewing And Changing Answers

The past decade has seen P&P tests being replaced by CATs within many large-scale standardized testing programs. However, many researches and applications on CATs have limitations, because most of CATs don't allow examinees to review and change answers. To examinees, not allowing reviewing and changing answers would bring them test pressure and affect their performance; To CAT test organizers, they don't allow examinees to review and change answers mainly because they are afraid that examinees would use Wainer strategy in review stage to obtain positively biased ability estimates. If we can provide such a solution that not only allows examinees to review and change answers but also be able to cope with Wainer strategy, the meaning is understood without explanation. There are few relevant studies on this field all over the world. In previous researches, researchers only recorded answers of review stage and used them to score, without considering answers of adaptive stage. This paper employed a new scoring method and tried to deal with Wainer strategy:V_αj = U_αj × Beta + U_α,j+m× (1 - Beta)Experiment 1 used Monte Carlo method to simulate CATs that allow reviewing and changing answers, aiming to examine the influence of different Beta values on ability estimation. Simulation results showed: (1) comprehensively taking the answers before and after review into account could produce more accurate ability estimation, and the best accurate estimates occurred when Beta=0.5; (2) most examinees changed answers; only a very small percentage of answers were changed; more answers were changed from wrong to right than from right to wrong. Experiment 2 used simulation data to evaluate the effectiveness of the Wainer strategy and tried to cope with Wainer strategy using new scoring method. Simulation results showed: (1) when using new scoring method, the ability estimates generated by Wainer strategy CAT obviously diverged from the true ability values. Moreover, the bias increased as true ability value increased; (2) when Beta=0, comprehensively taking the EAP and MLE ability estimates of Wainer strategy CAT into account could roughly deal with Wainer strategy.