A Comparison Of CD-CAT Item Selection Strategies Based On Testlet And Its Application

The computerized adaptive test(CAT)developed based on item response theory(IRT)can significantly save time of respondents and maintain good test accuracy.Compared to traditional CATs,the cognitive diagnostic CAT(CD-CAT)provides richer diagnostic information to the respondent and facilitates the design of targeted remedial instruction.In practical psychological or educational measurement,it is often the case that multiple items share the same stimulus,and the set of items sharing the same stimulus is called Testlet,e.g.,a reading comprehension,a graph,a table containing several questions under it.Testlets are widely used in a variety of tests,such as the English 4-6 exam for Chinese college students,the National Unified Examination for Graduate Students,and large tests such as the PISA and GRE in foreign countries.These testlet tasks are closer to the real situation,and the integration of knowledge,skills and abilities is required to complete them.Thus,they can provide more realistic and rich information.Most of the current researches on CD-CAT assume that the questions are independent of each other,i.e.,they do not take into account the questions in the testlet design.In this study,we develop a CD-CAT based on the testlet cognitive diagnosis model to extend the application of CD-CAT(TCD-CAT).In this paper,we develop TCD-CAT based on the testlet-DNIA model and compare the estimation accuracy and item exposure rate of traditional item selection strategies in the case of testlets.And add item exposure control methods to TCD-CAT to explore the performance of item selection strategies and item exposure control methods.The performance of the item selection strategies is compared in terms of estimation accuracy and security of the item pool;The security metrics of item poor include Test Overlap Rate(TOR),uniformity of question bank usage(χ^2),Underused Item Rate(UIR)and Overused Item Rate(OIR).The results show that TCD-CAT performs well for testlet tasks,and MPWKL performs the best among the item selection strategies.There is a significant improvement in the overall estimation accuracy and item usage of TCD-CAT after item exposure controlling.Finally,this article applied TCD-CAT to an English reading comprehension test,and the results showed that TCD-CAT was able to significantly reduce the number of items students required to answer.Meanwhile it had a cognitive diagnostic function that was useful for understanding the attribute patterns mastered by different groups of students and provided guidance for future remedial learning.The above findings suggest that TCD-CAT is feasible and can better handle the situation of having testlets in the item pool.Today,when there are fewer and fewer items satisfying partial independence,TCD-CAT has good development and application prospects,but research on testlet-CAT under cognitive diagnosis is still rare,and further theoretical research and practical applications are needed.