The Research Of Reviewable CD-CAT

Based on each examinee's answers of previous items, computerized adaptive testing(CAT) selects items and aims to measure examinee's actual ability more efficiently and accurately than paper and pencil-based tests.Combining cognitive diagnosis with computerized adaptive testing, cognitive diagnostic computerized adaptive testing(CD-CAT) aims to more efficiently and more accurately diagnose examinees' mastery status of a group of discretely defined skills, or attributes than paper and pencil tests.While it is a natural thing for examinees to review their answers and possibly change them in paper and pencil-based tests, the same thing is less common to happen in most CATs and CD-CAT since it could deteriorate the measurement efficiency. The absence of review opportunities on operational CATs and CD-CAT creates a dilemma for test developers as examinees need to review and change answers during the test in order to achieve more accurate estimates of their true ability. Researchers on reviewable CAT(RCAT) mainly focus on three aspects, namely changing the test design, improving the item selection strategy and building models.Item Pocket(Han, 2013) is a method of reviewable computerized adaptive testing(RCAT) and may expect a good prospect and application.This method provides test takers with IPs into which they can place items for later review and response change. Test takers can skip answering items by putting them in the IP. Once an item is placed in the IP, a test taker can go back to it anytime during the test until the test taker submits his or her final answer for the item. but the shortcoming of IP is that the capacity is not easy to control, if the capacity is too large that will results a comparatively large estimation error. Based on IP method, the study proposes a new IP method called modified IP(MIP), employing a new scoring method in IP.Compared with IP, Stocking(1997) design cause greater restrictions for examinee behavior. In Stocking design 1, examinees are instructed in advance that they will be permitted to revise answers to fixed number of questions, the revised answers do not participate in the adaptive selection items. Under Stocking design 2, the testing is divided into separately sections and examinees are informed in advance of testing that they will be permitted to revise answers to questions only within a section. The advantage of design 2 over design 1 is that design 2 simultaneously restricts examineecontrol over the actual item presented because revised responses from previous sections influence the section of items in subsequent sections.Cognitive diagnosis computerized adaptive testing(CD-CAT) was a further development of the CAT, but they were very different in some ways. In order to verify the above methods in(Reviewable Cognitive Diagnostic Computerized Adaptive Testing, RCD-CAT), two Monte Carlo simulation studies with different experimental conditions were conducted here, the interim and final states of knowledge were estimated using the maximum likelihood estimation(MLE) method, a group of 5,000 examinees were simulated for this study, and the tests were then created from an item pool of 300 items. These experimental conditions were cognitive diagnosis model(DINA and R-RUM), the number of attributes(5 and 7), item selection strategies(KL, PWKL, HKL and MPWKL), and the fixed test length CD-CAT(10 and20 items respectively).Monte Carlo simulation results show that:Firstly, Compared with the traditional CD-CAT, the reviewable method(MIP)proposed by the article did not lose the diagnosis accuracy and bank exposure rate, at the same time, allowing students to change the answer, which accord with the general students answer behavior, reduce the burden of students examination and anxiety levels. It was more likely to be accepted by the public.Secondly, When using the DINA model, MIP and IP methods had very similar classification accuracy.In addition, while using R-RUM model MIP method had higher classification accuracy, which indicated that MIP effects depend on the answering probability distribution.Thirdly, In any experimental conditions, Stocking design had the highest classification accuracy, and Stocking design 2 was slightly better than the Stocking design 1. From simulation results, we found that Stocking design had a much better prospect in the RCD-CAT.In a word, RCD-CAT is more consistent with traditional examination habits, in addition, it can also improve classification accuracy. This study will help to provide theory and method support for future research and practical application.