Research On Nonparametric Cognitive Diagnosis Method And Item Selection Strategy Of Nonparametric CD-CAT

Cognitive diagnostic computerized adaptive testing(CD-CAT)combines computerized adaptive testing(CAT)with cognitive diagnostic(CD).CD-CAT can provide fine-grained and diagnostic results efficiently and quickly by customizing individual tests for candidates.Compared with the traditional paper-and-pencil test(P&P),CD-CAT can guarantee higher measurement accuracy while setting fewer questions.Non-parametric CD-CAT has the characteristics of convenience efficiency and no need for model fitting estimation.Like the parametric CD-CAT,the test process usually consists of six parts: the selection of the initial item,the participant’s answer,the estimation of the participant’s knowledge state,the item selection strategy,the termination of the test,and the result report.Among them,the estimation of the participant’s knowledge state and the item selection strategy are the core parts that affect the accuracy of the test.In the non-parametric CD-CAT,the non-parametric cognitive diagnosis method only needs to set the Q matrix to accurately estimatethe participant’s knowledge state;the non-parametric item selection strategy does not require parameter calibration,and it also performs well in small-scale classroom assessments.(Chang,Chiu & Tsai,2018).At present,non-parametric cognitive diagnosis methods and item selection strategy are mainly developed based on haiming distance(HD)and manhattan distance(MD).HD and MD are currently the two most widely used vector distance calculation methods,which are equivalent under the 0-1 scoring method.Educational and psychological test data are not limited to 0-1 scoring items.Compared with 0-1scoring items,polytomously scoring items can provide more detailed information.However,there are very few researches on non-parametric cognitive diagnostic methods and item selection strategy that are targeted at polytomously scoring items.In a word,the non-parametric method provides the possibility for cognitive diagnostic assessment to go to the classroom,and it provides method and technical support for front-line teachers to apply classroom assessment.Therefore,it is of great significance to study the non-parametric cognitive diagnosis methods and item selection strategy of polytomously scoring.In view of this,this article carries out three studies:In the first study,two succinct non-parametric cognitive diagnostic methods are proposed for polytomously scored items: Lance Distance Discrimination(LDD)and Jaccard Similarity Discrimination(JDD).Through simulation researches,LDD and JDD are compared with MDD,G-WMD and G-GDD under a variety of experimental conditions(probability of slip,test length,attribute hierarchical relationship and number of attributes).The results show that: First,in the non-parametric discrimination method,LDD and JDD have the highest discrimination rate,followed by MDD,G-WMD and G-GDD.Second,When the attributes are more and the test length is shorter,the accuracy of each method is lower.Third,under different experimental conditions,the variation range of the accuracy of G-GDD,LDD and JDD is less than 0.3,which is more stable than G-WMD and MDD.Fourth,it is found that the hierarchy and the number of attributes have a significant interaction effect on PMR.In the second study,in order to thoroughly test the suitability of the three discriminant methods,the validity of the three discrimination methods is examined in empirical data.The results show that: LDD,JDD and MDD have good external validity and can be used as effective evaluation tool.In the third study,a non-parametric CD-CAT item selection strategy are proposed for polytomously scored item.First of all,LDD,JDD and MDD are combined with RMPI to control item exposure.Three kinds of mixed scoring non-parametric CD-CAT topic selection strategies are constructed as RMPI-LDD,RMPI-JDD and RMPI-MDD.Secondly,the NPS method of 0-1 scoring is extended to the non-parametric CD-CAT of polytomously scored item,which is denoted as MNPS(modified NPS).Finally,This research clarifies the algorithm steps of item selection strategy,sets up a variety of experimental conditions(test length,attribute hierarchical relationship and Q matrix correctness)in the mixed scoring test,and compares with classification accuracy and test safety of four selection methods(RMPI-LDD,RMPI-JDD,RMPI-MDD and MNPS).The results show that: First,regardless of the correction of the Q matrix,MNPS has the highest accuracy rate under all experimental conditions,followed by RMPI-LDD and RMPI-MDD,and finally RMPI-JDD.Second,RMPI-LDD,RMPI-MDD and MNPS have higher test security.Third,RMPI-LDD and MNPS perform better,and RMPI-MDD is easily affected by the length of the test and the attribute hierarchy.Fourth,regardless of the level structure,When the test is shorter and the Q matrix error rate is higher,the accuracy of each method is lower.And RMPI-JDD has the largest decrease.In general,under the non-parametric CD-CAT framework,the cognitive diagnostic methods and item selection strategies developed for the polytomously scored item have achieved better results.In the future,it can be considered to further strengthen the construction of revised indicators based on the non-parametric KS discrimination method and Q matrix,and the research and development of the non-parametric CD-CAT system.