Research On Development,Validation And Application Of Quality Index Of Cognitive Diagnosis Item

Cognitive diagnosis item quality index(abbreviated as quality index)have many applications in the cognitive diagnosis,such as improving diagnosis accuracy,controlling the exposure of item banks to reduce the risk of item bank leakage,and developing new diagnosis testing methods.However,the current research and application related to quality index have the following four problems:First,the existing quality index is effectiveness has only been verified on simplified models,while its performance in quality evaluation on saturated models is unknown.By analyzing the mathematical expressions of quality index,it can be seen that some quality index in the saturation model tend to ignore the performance of the project in correctly classifying the partial mastery group,thereby failing to fully evaluate the item quality of the saturation model;Secondly,most of the existing quality index are developed based on dichotomous scoring item,which cannot be applied to polytomous scoring item;Moreover,the effectiveness of universal index suitable for arbitrary scoring methods for polytomous scoring item quality evaluation remains to be verified;Finally,diagnosis test assembly as the most important application of quality index,currently only obtains relevant verification in the case of dichotomous attribute and item dichotomous scoring tests,which is difficult to conform to actual test situations,such as polytomous attribute,dichotomous scoring item and polytomous scoring item mixed test forms.This paper carries out three studies on the construction and application of quality indices.In the first study,aiming at the deficiencies of some quality indices in the multiple response probability model,the extended CID and extended NID(abbreviated as GCID and GNID)are proposed.In addition,from the perspective of diagnostic residual,the ideal cognitive diagnosis quality index(abbreviated as IDI)are proposed,and the properties of three new quality index are discussed from the general formula of different quality index.In order to verify the effectiveness of the new quality index,R language is used to simulate the calibration correlation between the new quality index and the existing quality index.The results show that:(1)Under the DINA model,there is a high correlation between the three new quality indices and the existing indices.(2)Under the GDINA model,the three new quality indices have high correlation with CDI and ADI index,but low correlation with CID and NID index.In study two,Two quality indices verification methods were used:the cognitive diagnosis automated test assembly(abbreviatedcator as CD-ATA)and the cognitive diagnosis computer adaptive test(abbreviated as CD-CAT).In the GDINA model,the ability of three new quality indices to explore the static and dynamic quality of the item was verified respectively.Control a variety of variables that may affect the results,and use a variety of evaluation factors such as the accuracy rate and the security of the item bank.The results show that:(1)On the whole,the three new quality indices can effectively find high-quality items in the item bank,and achieve the item quality identification ability similar to or even better than CDI and ADI.(2)In terms of identifying the static quality of the item,the three quality indices are more inclined to identify the item with fewer attributes as a high-quality item,which is consistent with the existing quality index.When three new quality indices are used to group papers,the accuracy of the new quality index is similar to or even higher than the existing index.(3)In terms of identifying the dynamic quality of items,three new quality indices can effectively find items with high dynamic quality,and integrate them into the topic selection method to improve the accuracy rate of cognitive diagnosis.The shorter the test length,the higher the quality of items,the more obvious the improvement.In the third study,we explored a more practical situation of polytomous attribute and mixed scoring test.In order to develop the CD-ATA method suitable for such test scenarios,the existing MCDI and MADI methods are first expanded to obtain the expanded GMCDI and GMADI;Secondly,carry out polytomous scoring expansion on the IDI quality index to obtain the polytomous scoring IDI(abbreviated as PIDI),and correct the PIDI to obtain the modified PIDI(abbreviated as MPIDI).The above paper forming method is applied to the CD-ATA with simple dichotomous scoring item test and polytomous scoring item test,and mixed dichotomous scoring item and polytomous scoring item test.The results show that:(1)polytomous scoring item are more conducive to the evaluation of attributes at polytomous levels.(2)GMCDI and GMADI are not affected by the attribute structure and test type in CD-ATA and can achieve high diagnosis accuracy.(3)PIDI and MPIDI are more suitable for dichotomous scoring test and polytomous scoring item test,but not for mixed scoring.Under certain conditions,PIDI and MPIDI can obtain the best accuracy.