Research On The Validation Method And Application Of Q-matrix In Polytomously Scored Cognitive Diagnosis Assessment

Q matrix plays an important role in the estimation of model parameters,and the diagnostic classification of test-takers.At present,most of the existing research on Q matrix validation is based on the binary case where the response vector element is 0-1 Some studies have shown that scoring the responses of the subjects at multiple levels can provide more diagnostic information to the subjects,and the tests in China are mainly scored at multiple levels.Based on this,this study intends to explore the method of Q matrix validation in the case of polytomous scoringBased on the response data of the subjects,this paper intends to investigate the validation of Q matrix by using the nonparametric method and the parametric method,respectively.Consider a sequential process model,which can address multiple steps with polytomous items,and propose two methods to address the issue of Q matrix validation.Among them,the non-parameterized method considers to improve the method of Chiu(2013)(called RP method),while the parameterized method mainly extends the method of Liu,Xu and Ying(2012)(called SP method),so that they can be applied to the polytomous scoring cognitive diagnosisIn order to evaluate the performance of the improved algorithm,we carried out a series of monte carlo simulation studies,taking into account the following factors:the number of subjects and the proportion of errors in the Q matrix.The performance of the extended method in the empirical data is further evaluated.Three studies were conducted:study 1 is based on the nonparametric method to validate the Q matrix when the respondents' answers are polytomous scoring;study 2 is based on the parametric method to validate the Q matrix when the respondents' answers are polytomous scoring;study 3 is the empirical research.The results show that(1)As for RP method,when the sample size is large and the error proportion is 5%,the method has a high correct estimation rate of Q matrix;When the sample size is relative small and the proportion of the wrongly specified elements in the initial Q matrix is high,the correct estimation rate of the Q matrix is about 50%.In addition,the correct estimation rate of attribute vectors in subsequent steps decreases gradually with the steps going on.(2)In the case of large sample size and error ratio of 5%,the SP method has a high correct estimation rate of Q matrix.When the proportion of small samples and error tables is high,the correct estimation rate of Q matrix is about 60%,and the correct estimation rate of vector elements in subsequent steps decreases gradually after the first step is eliminated.(3)In the empirical data research,two methods are used to validate Q matrix,and there are some differences in the correction results.In general,compared with the initial Q matrix,the two methods are more inclined to "overestimate" some properties when estimating Q matrix under polytomous scoring conditions.