Q-matrix Validation Method Based On RRUM Model

Cognitive diagnosis aims to assess the specific knowledge and skills that candidates possess.In parametric cognitive diagnosis,it is necessary to use cognitive diagnostic models to infer students’ knowledge mastery details.Q matrix is a core component of cognitive diagnosis,which has an important impact on model parameter estimation,diagnostic classification,and model data fitting.Currently,the research on Q matrix verification methods is basically based on the DINA model and the GDINA model.Among the many models developed,the RRUM model is also a popular model in cognitive diagnosis.The Q matrix verification methods based on the RRUM model are relatively few,and they are not systematic and in-depth enough.Therefore,this study intends to carry out validation research on the Q matrix under the RRUM model and propose corresponding validation methods.Regarding the definition criteria of the Q matrix,referring to the method of de la Torre(2008),this study explores the variation patterns of the project parameters(n*and r*)of the RRUM model under different types of Q matrix errors.Based on the variation patterns,different empirical standards(Δπcv and rcv)are selected.In order to evaluate the effectiveness of different empirical criteria on Q-matrix recovery in two common search algorithms(item-attribute space search algorithm and sequence search algorithm),we conducted a series of Monte Carlo simulation studies and empirical data analysis,considering two real Q-matrices and four influencing factors:number of participants,project quality,error ratio of Q-matrix,and distribution of attribute mastery patterns.Three specific studies were conducted:Study 1 explored the variation patterns of project parameters in the RRUM model under different Q-matrix error types,and provided corresponding mathematical proofs;The second study explores the Q matrix restoration effect under different empirical standards in two search algorithms through simulation research,as well as the impact of four factors on the Q matrix restoration effect;The third study is empirical data application research,further demonstrating the process of using the method proposed in this article.The research results indicate that:(1)In the sequence search algorithm,the empirical criteria of π*and r*(Δπcv and rcv)are considered simultaneously.There are a total of four combinations of empirical criteria.The results show that,on the whole,The Q matrix recovery effect is better under the standard Δπcv=0.05&rcv=0.7.(2)In the project attribute space search algorithm,only the empirical criterion(rcv)of r*is considered,and the results show that rcv=0.7 compared to rcv=0.8 performs better,with a higher Q matrix recovery rate.(3)Regardless of which search algorithm is used,the impact of various factors on the recovery rate of the Q matrix is consistent.In terms of attribute mastery mode distribution,the Q matrix recovery rate under uniform distribution is always better than that under multivariate normal distribution.The quality of the project has an important impact on the recovery of the Q matrix.When the project quality is low,the recovery rate of the Q matrix improves as the number of subjects increases.The error ratio of the Q matrix has little impact on high-quality projects.(4)When the item length and number of attributes are small,the Q matrix recovery rate and computational efficiency of the two search algorithms are equivalent.As the item length and number of attributes increase,the advantages of the sequence search algorithm become more apparent.When the attribute mastery pattern distribution is multivariate normal distribution,the recovery rate of the Q matrix obtained using the sequence search algorithm is higher than that using the item attribute space search algorithm.(5)In empirical data research,Δπcv=0.05&rcv=0.7 to verify the Q matrix.Compared with the initial Q matrix,the modified method tends to "underestimate" some properties of the Q matrix.