The Development Of Generalized Online Calibration Methods In CD-CAT

Item replenishing is critical for maintaining item bank in cognitive diagnostic computerized adaptive testing(CD-CAT).Under the framework of CAT or CD-CAT,online calibration is an important means for item replenishing.Currently,there are only a few online calibration methods(e.g.,SIE and JEA)for CD-CAT,which are available to calibrate Q-matrix and item parameters.However,these methods can only be used directly to some special cognitive diagnosis models(CDMs)(i.e.,the Deterministic Input,Noisy and Gate Model,DINA)in CD-CAT.That is to say the existing methods cannot be directly used in other more general CDMs.The current study aimed to develop generalized online calibration methods for calibration of Q-matrix and item parameters,which can be employed in a wide class of CDMs including both reduced and saturated CDMs.Based on the existing online calibration methods SIE and JEA,the SIE-R,SIE-R-CY,JEA and JEA-R-CY methods that take full consider of item prior information are proposed in this study.Moreover,the SIE-R-BIC and JEA-R-BIC methods are proposed based on the consideration of model complexity.Furthermore,the proposed method RMSEA-N is constructed according to the consistency between the expected distribution and observed distribution of response probabilities.Then,two simulation study was conducted to verify the feasibility and effectivity of the proposed methods in DINA,DINO,ACDM,RRUM and G-DINA models,and to explore how to the three factors,item parameter ranges(U[0.05,0.25] and U[0.1,0.4]),online calibration designs(adaptive design and random design)and sample sizes(500,1000,1500),effect the calibrated precision of the online calibration methods.Results indicated:(1)The proposed methods generated both precise Q-matrix calibration and accurate item parameter calibration,which has wider applicability than the existing methods.(2)The smaller the value of item parameter range is,the higher the correct estimation rate of the attribute vector and the item parameter is;The effect of online calibration design on Q-matrix calibration precision is small,but the effect of online calibration design on item parameters calibration is negligible;Larger sample size yielded better Q-matrix calibration precision and item parameter calibration accuracy in most proposed online calibration methods.(3)The proposed SIE-R-CY method has the best item calibration accuracy,followed by the proposed RMSEA-N method in DINA and DINO models;The RMSEA-N and SIE-R-BIC methods have their own advantages based on different item parameter ranges and sample sizes in ACDM,RRUM and GDINA models.Finally,the defects and shortcomings of this study are discussed,and some directions of future studies are provided.