Item Selection Methods In Multidimensional Computerized Adaptive Testing Adopting Polytomously-scored Items

Multidimensional computerized adaptive testing(MCAT)features a combination of computerized adaptive testing and multidimensional item response theory(MIRT)which shows great potential to obtain diagnostic information from examinees' item responses accurately and efficiently,with its goal of extracting as much information as possible about the multiple abilities required.Polytomously-scored items have been broadly used in a variety of tests for their advantages of providing more information and testing complicated abilities and skills.It is promising to develop MCAT adopting polytomously-scored items(PMCAT).However,most item selection algorithms used in MCAT so far are derived for dichotomously-scored M3 PL or M2 PL model,might not be appropriate for the polytomous items.The purpose of this study is to discuss the proper way to extend the item selection algorithms used in MCAT for PMCAT.Several promising item selection algorithms used in MCAT are modifies for PMCAT,and a new selection method is proposed to improve the existing selection strategies.This study also explores how the number of dimensions and the correlation among attributes affect the accuracy and security of PMCAT.Several well-used MCAT select criteria--including FI-based D-optimality,A-optimality,E-optimality,and each of their Bayesian versions;conventional KL information based KL index(KI)method,posterior expected KL information method(KB);and methods based on KL distance between subsequent posteriors,containing the KLP method,mutual information method(MUI)and the CEM method were extended to suit polytomously-scored items.Then the CEM was modified by replacing the fixed prior by the varied current posterior(7)(8)1 1?| Uk k?--in the calculation of posterior(7)(8)1?| U,uk jkk?-.Then a Monte Carlo simulation study was conducted to demonstrate the feasibility of the PMCAT,and to compare the performance of these proposed methods.Another Monte Carlo method was used to explore how the two factors-the number of dimensions(p=2 and 5),and the size of correlations between dimensions(r=0,0.2,0.5 and 0.8),effect the estimated precision of ability and the item explore rate.The polytomously-scored model used in this study was the multidimensional graded response model(MGRM),and EAP was used as the latent trait estimation approach when the test was ongoing with the stopping rule of fixed length was used.The simulation results showed that most of the extended select criteria for PMCAT were feasible,except KI produced relatively larger estimation error.The simulation results showed that most of the extended select criteria for PMCAT were feasible and the new item selection method for PMCAT performed well.Combine with the security of the pool,the proposed MCEM was the ideal of all,for it gained the lowest item exposure rate and had a relatively high accuracy.And the number of dimensions and test items and the correlation among attributes effected the ability estimation accuracy and item exposure rate.The findings suggested:(1)With the exception of KI method,most of the item selection strategies gained bigger estimation error while the number of dimensions increased.The estimation accuracy of KI method increased with dimensions from 2 to 5.(2)The correlativity had an impact on estimation accuracy only when the correlation between ability dimensions was higher than medium.The estimation error of KI method dropped dramatically while the correlation becomes larger.(3)The more the dimension,and the greater the correlation between ability dimensions,the lower the item exposure rate would be.Especially for A-optimality method,it gained highest item exposure rate in the 2 dimensional case,while owned the lowest item exposure rate in the case of 5 dimensions.Polytomously-scored items are widely used in Likert-type scale in psychological inventories and are also highly applied in the achievement context.There will be a wild area for the application of MCAT adopting polytomous items.The extension of item selection methods for PMCAT can be referenced to theoretic study and practical application.